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ABSTRACT v 

Shrinking training budgets pose a serious problea to 
those confronted with the present and future challenge of providing 
coapetent Air Force technicians for increasingly technical positions 
in a aodern Air Force. One promising solution to this problea has 
been to harness the capabilities of the coaputer as an instructional 
training device. To be cost-effective, computer-based instruction 
aust aariaize individual student attainaent of training objectives, 
while simultaneously minimizing training time and costs. Adaptive 
Instructional Models (AIM) constitute the means by which effective 
training can be accomplished with a minimum expenditure of student 
time and instructional resources. The report describes the purpose 
and function of AIM. Additionally, seven adaptive instructional 
models are analyzed, and recommendations as to model application in 
Air Force technical training courses are made. (Author) 
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SUMMARY 



Problem 



Objectives of the Phase I research were to (I) provide in a single source, a comprehensive review and 
anlysis of state-of-the-ait developments in adaptive instructional models and(2);ecommend which adaptive 
instructional models as a subsystem of the Advanced Instructional System (AIS) were suitable for 
implementation within three Air Force technical training courses (Precision Measuring Equipment, 
Inventory Management, and Weapons Mechanic). 



Literature searches were conducted to identify and document current trends in both the research and 
development of adaptive instructional models. Professional contacts and surveys of facilities currently 
exercising computer-managed adaptive instructional inodels were made. Based upon these information 
sources and familiarity witli student characteristics associated with the three designated Air Force technical 
courses, specific recommendations were made for optimizing, incorporating, and implementing 
instructional models within the AIS; 



Five Adaptive Instructional Models (AIM) useful for specific instructional tasks were recommended 
for immediate implementation within the AIS. Briefly, the five adaptive models and their associated tasks 
were (1) DrUl and Ptactke-inctedi^s student proficiency and speed; (2) Q>^cept Acquisiiion-hcdiiates 
concept attainment by varying the sequence, amount, and kind of examples; (3) Cdmplex 
Tutorial-^Toyidts the student with strategies with which to master rule-learning and problem-solving; (4) 
Algorithmic Regression -dei^it a plan of instruction for each student in the form of a prescription, assigns 
resources, provides incentives, and monitors outcomes for input into the next individualized prescription; 
(5) Dynamic Programming-^ a, mdisiet instructional model which is capable of incorporating previously 
met\tioi\ed models in order to optimize student progress/ proftci^a1cy, aKd instfuctldnaTirS^ ^- ^^ ^ 

Two additional models were analyzed and recommended for further research prior to field 
implementation within iho AIS. These models include ( 1) Natural Langyage Processing and (2) Automaton 
Models. 



Five state-of-the-art Adaptive Instructional Models were recommended for inclusion within the AlS. 
A future technical report (Phases II & III) shall report: (1) findings of computer simulations of three of 
these models, and (2) a slep-by-step guide to how these models may be used to maximize student 
performance while minimizing student training lime and instructional resources expended. 



Approach 



Results 



Conciiisions 
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PREFACE 



This report documents a comprehensive analysis of state-of-the-art developments in 
adaptive instructional models. Recommendations as to the suitability of numerous 
instructional models for application in Air Force training have been provided. 

Research was accomplished in support of Project 1193, Advanced Instructional 
System (AIS), Task 0B05, Analysis and Development of Adaptive Instructional Models 
for IndWidualized Technical Training. Mr. Joseph Yasulake was the Project Scientist, and 
Dr. Gerard M. Deignan was the Task Scientist. 

Research contained herein was conducted under the provisions of Contract 
F33615-72-C-1277 by Dr. Duncan N. Hansen and his staff at the Center for Computer 
Assisted Instruction, Florida State University. Subsequent technical reports under this 
contract ^all provide computer simulations of three adaptive instructional models and 
their technical training applications in Air Force settings. 

Grateful appreciation is extended to two individuals for their personal 
contributions during the course of this research. Dr. Marty Rockway, Technical Director, 
made sound scientific suggestions in several portions of Ac manuscript. As evidenced by 
the original statement of work, Dr. Pat-Anthony Fcderlco provided the research impetus 
and initial technical monitorship in the formative research stages of this contract. The 
summary and overview was prepared by Dr. Gerard M. Deignan, Technical Training 
EHvision, Air Force Human Resources Laboratory. 
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THE ANALYSIS AND DEVELOPMENT OF AN ADAPTIVE INSTRUCTIONAL 
MODEL(S) FOR INDIVIDUALIZED TECHNICAL TRAINING-PHASE I 



I. THE NATURE AND ROLE OF ADAPTIVE INSTRUCTIONAL MODELS 

Ovetview 

Adaptive Instructional Models (AIM) provide the means by whl^fe instructional task factors, 
materials, and resources are continuously tailored or adapted to match the chan^ng instructional needs, 
skilis, and interest motivations of Individual students-and seeks to do so in a cost-effective manner. Indeed, 
providing more cost-effective training by maximizing individual student attainment of training objectives, 
while simultaneously minimizing the completion time and costs of such training, is the prlmaiy contem of 
this project. 

To provide a more meaningful context for discussing the AIM, it might be helpful to describe, briefly, 
the larger Advanced Instructional System (AIS) within which the AIM subsystem is embedded. 

The AIS is a prototype, computer-based, individualized training system and research facility designed 
to improve and maintain the cost-effectiveness of technical training. Individualization of technical 
instruction, computer management of instructional resources, and evaluation of Instructional strategies are 
some of the AIS provisions anticipated to contribute considerably to the effectiveness and efficiency of 
techmcal training. Additionally, to exploit the utility of future instructional innovations, the AIS will 
serve as an evaluative testbed for promising instructional technology. 

Initially, the AIS shall be implemented in three Air Force technical courses: (1) Precf$ion Measuring 
Equipment, (2) Weapons Mechanic, and (3) Inventoiy Management. At the conclusion of this stage, the AIS 
shall operate as a totally Integrated computer-based system, capable of training approximately 2,100 
students in the three coursed \yith a 25-percent average reduction in training time. Furthermore, time 
savings shall be accompanied by t|aining proficiency equal to or better than former, non-AIS graduates. 

For the purpose of discussion, the AIS migjit be compared to the structiires and^ 
human body, wherein the AIM*s constitute the integrated "brain" of the AIS. This particular brain 
assimilates and stores information about each individual student's characteristics, and it compares such 
information to current instructional tasks. Based upon this stored Information, the brain scans the available 
instructional material, media, and other resources to determine the level of content to present and the best 
method of presenting it. The brain then composes a strategy which attempts to maximize sttldent 
performance with a minimum expenditure of student time and instructional resources. 

Because this brain continuously monitors and manages not only student performance, but 
instructional resources availability as well, it is capable of learning more about each student, to include 
which instructional methods, materials, niedia, and incentives are performance effective. Hence, 
instructional strategy effectiveness becomes prc^ressively more, accurate. Furthermore, if certain 
instructional equipment is predicted to be in ^ort supply at a particular point in time, the brain may 
prescribe an alternate instmctional path which permits equipment use at a more appropriate time. 
Adaptability to constraints of limited instructional resources and the prefer scheduling of resources are* 
major determining conditions of instructional slrate©' effectiveness and rdated time savings. 

The objectives of the AIM*s contribute directly to the goals of AIS, namely, individualization of the 
training process, computer-management of resources, use of cost-effective multimedia approaches, and 
training systems modularity. Specifically, AIM contributes to the following AIS goals: 

♦ Adaptivtntss- AIM provides for individualization of the training process based on student 
characteristics and specific training strategies, 

• Flexibility- AIM provides training alternatives in content, media, and personnel assignements, 

• Expandability- AM provides an updating mechanism by ^ich more proficient instruction can 
occur. 

♦ Modularity- AIM provides alternative models v^kh can be utilized according to the appropriate 
match between training task requirements and specific individual differences. 
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• Cost Effectiveness- AIM attempts to optinii/e student motivation with incentive schemes that will 
lead to significant reductions in training time. 

The objectives of AIM can be stated as follows: 

• To provide a fine-grained monitoring of each student's performance 

• To provide a set of training decision rules that optimize students' motivation and progress 

• To provide a decision-allocation procedure that optimally assigns instructional media, material^ and 
incentive rewards according to each student's characteristics and performance 

• To provide predictions of performance and time parameters for each student 

• Toprovidifor the scheduling of all instructional resources so as to minimize cost. 

An extensive review of prior research and theoretical literature has led to an initial classification of 
the adaptive models into seven groups As is described in this report, five of these mode! groups are 
recommended for immediate implementation witliin AIS, The remaining two mode! groups, being more 
abstract, are recommended for additional research investigation prior to incorporation within the AfS 
program. The essential features of all seven adaptive models are discussed in the foil owing paragraphs. 

The first model, Drilland-Praciice, has two primary goals, namely, (1) to improve tlie student's 
accuracy, and (2) to increase his speed of performance, Drill-and-praciice models primarily provide for 
appropriate problem selection and control of instructional presentation and/or student response rates so as 
to achieve their overall goal. The drill-and-practice models will typically be embedded in strategies for 
practice following demonstration, remediation, and review. These practice strategies are especially 
important in technical training environments. From an operational point of view, the drill-and-practlce 
model provides for a computer-based composition of a set of problems, appropriately graded and ordered as 
to dijfficulty . To a large extent, attainment of specified criterion levels of skill performancie depends upon 
proper sequences and amounts of practice and review. 

Concept Acquisition Models provide for the dynamic manipulation of factors related topretask arid 
-within-task-variable&so.as.to-facilitate learning. proces$ses..Pietask,Yarjabl^^^ 

ability, and learning styles which are used to manipulate both the concept-content and content difficulty 
levels. Withln-task variables provide for a systematic manipulation of positive and negative examples, the 
number of examples, the degree of prompting, and the nature of the correctional feedback process. Thus, 
the concept acquisition models provide for extensive adaptability appropriate to the diversity of technical 
concepts found within AIS. 

Complex Tutorial Models provide the strategies by which rule-learning and problem-solving behaviors 
can be achieved by students. The complex tutorial model utilizes a combination of multiple regression 
techniques and explicit decision rules to select instructional content, examples, and problems to compose 
individualized lesson sequences for a given student. The decision rules will relate such task characteristics as 
rule difficulty, problem difficulty, and/or example difficulty with the student's personal profile, which 
includes such variables as cognitive ability and learning style. For instmction and technical education, the 
complex tutorial models provide for appropriate sequencing of the amount of practice with rule statements, 
examples, and problems, especially as these are integrated into highly complex behaviors. 

Algorithmic Regression Models provide for individualized prescriptions based upon task, media, and 
learning variables. In essence, the regression-based learning prescription spells out a plan of instruction for 
each student. In addition, the Algorithmic Regression Models will assign appropriate incentive levels so as to 
optimize the motWational state of the learner. As an added feature, these models allow for optimal 
assignment of resources, given that one wants to maximize a common goal such as total progress of all 
students as opposed to maximizing the progress of an individual student solely. 

Dynamic Programming Models provide for a sequential or multistage decision piocess that can 
incorporate many tasks, media, student, and resource variables. The dynamic programming models convert 
these multistages into a series of single-stage problems which can then be optimized. Most importantly, 
dynamic programming models provide for a hierarchical nesting of other adaptive models, and, in essence, 
function as the master model for AIS. 

* In addition to (he above models, it was recommended that further research be directed toward 
extending A'c?/wn?^ Unguagc Processi^^g and Automaton Models. 
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Natural Language Models provide a conversational dialogue between (he system and the studcni. This 
dialogue requires a fuJl representation of the student*s language and mental processes. The 
interactive-contingent nature of the dialogue allov^fs for mutual information clanfication and stimulates the 
development of student competencies as he achieves masteiy oVer given training objectives. Natural 
language processing riiodcls should also prove helpful in extending student evaluations to counsehng 
processes within AIS, 

Autonuitlon Models provide foV the abstract theoretical representation of a student-training system; 
This representation is in terms of states that are defined by input and output relatlonsJiips. Prom a monitor* 
ingand prediction point of view, automaton models represent the most advanced theoretical developments. 
Unfortunately, their operational representation is still exceedingly limited. Consequently, basic research 
progress wflt be necessary pnor to full implementation within AIS. 

11. INTRODUCTION TO ADAPTIVE INSTRUCTIONAL MODELS 

Adaptive Instmclional Models (AlNVs) constitute means for individually prescribing, analyzing, and 
adapting instmclional materials, tasks, and resources to differences among students on such dimensions as 
ability and motivation. The ultimate goat is to train competent Air Force technicians in the minimum time 
at a cosl^ffective level. 

First, the AIM*s serve as a representation of its characteristics and operations of the Advanced 
Instructional System (AIS). In that sense, AIM plays both the role of a model and of a simulation. More 
explicitly, the proposed AIM shall be implemented as synthetic training sessions which provide "Monte 
Carlo*' data representative of student performance plus the training outcomes anticipated for AIS. Second, 
A!M*s represent the niathematica! tools whose formal processes and parameters allow for accurate 
predictions about the outcomes and adaplations invdved in the AIS methods of training. In turn, the 
simulation aspects of the AIM*s provide guidelines which can be used in the design and Implementation of 
the AIS computer managemcni system, Finally, AIM, when programmed, will provide code for the AIS 
operation as well as the first concrete, explicit example of the AIS performance recording data base 
necessary for contingent adaptive training procedures. 

The payoff from AIM for the AIS project can be viewed in both short-and long-tenn beneftts. For the 
short term, the principal benefit wfll be the clarification of adaptive training processes. This clarification 
wfll result from the detailed literature search and model identification provided within this document. 
' Anotlier benefit is that classes of adaptive models will be identified in terms of their purpose, tlicir formal 
structure, and their potential payoffs. These elements are presented in Sections 11 through IX to allow for 
an initial consideration of which adaptive models to pursue within AIS. As indicated previously, the 
simulation aspects of this AIM study will provide design guidance as well as representation of actual AIS 
computer operations. 

As for long-temi benefits, AlM's provide tlie conceptual basis for this individualized form of technical 
training. Finally, AIM provides for a coherent paradigm for the multivariate requifcn>ents that functionally 
represent student characteristics, instructional modes, task characteristics, training decision processes, and 
the allocation of instructional resources. 

Concept of AIM Within AIS 

The AIM will play a general coordinating role in the AIS, It can be anticipated that the AIM will 
structure learning prescriptions based on student and task characteristics and monitor all siudents* training 
activities in order to provide input to the adaptive process. In addition, there is the potentbl for monitoring 
and optimally assigning learning resources, such as instructors, media devices, simulators, or Instructional 
' software. Thus. AIM is inextricably involved in all phases of the AIS operational environmcjit. 

AIM CharaciCAStics 

The Adaptive Inslructiofial Models will have foiir properties that promote the individu -i ^nof the 
training procc adaptivencsv amtingency, mediation, and cybernetic* 
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AIM is adaptive, in that the training (Process will be individually tailored to ^ch student. In 
operational terms, the training decisions will be made by continually choosing among Instructional 
alternatives as a function of differential student characteristics, as measured by the AIS training process. 
The concept of adaptiveness will include the features of selectiveness since each student will be presented! 
with information according to his needs in light of the terminal objectives, sequenced because the materials 
wfll be presented in an optimal sequence for eath student, zni paced since the student wUl be provided 
with a rate score commensurate with his prior performance and his learning characteristics. In addition, 
adaptiveness will Include the concept of individually-prescribed media, amount and type of review, and use. 
of remedlatlonal material. 

In reference to the second feature, contingency, AIM will provide relationships which will consider 
who is being taught, vvftar is critical in the subject matter^ and tow the teaching is to be done/ This wSU 
include st raters by which student characteristics arc matched with a catalog of training alternatives under 
the control of computer-based algorithms so as to prescribe optimal sequences. In addition, contingency 
will include the concept of individually-prescribed incentives according to performance related incentive 
schedules, as well as opportunities to branch or re-enter learning sequences according to identified levels of 
mastery • 

The third general characteristic concerns the mediation process, which will include a wide range of 
media and learning formats configured to optimize the information flow according to specifiable 
subject-matter maps. In essence, the subject-matter maps will be defmed in terms of task characteristics and 
will lead to an optimal matching of training resources in light of task features and student characteristics. 
Mediation will include the concept of appropriate media matches as well as individual and small group 
instmction. Where appropriate, the student may also be assigned to individual counseling sessions designed 
to facilitate the lean\ing process. 

An empirical feedback procedure which uses student data from established criterion measures to 
redefine parameters of th^ strategies and their embedded decision rules characterizes the cybernetic feature. 
Each student will be continuously monitored by AIM so that his profile identifies his current status as well 
as his best performance within various instructional strategies. The feedback of su^ess and failures will not 
only be recorded for individual students but also will be aggregated so as to improve the overall modeling 
process for new groups of students. This continuous updating of student performance will improve the 
accuracy of both the individual learning prescription and the modePs predictions of optimal learning 
sequences. Thus, AIM provides data which wfll cybernetically improve the performance of the model itself, 

Computer Managed Instmctlon (CMO Model 

As a framework, AIM must provide a proper flow so that students may be prescribed learning tasks in 
an individualized sequence. As presented In Figure 1 , this adaptive instnictlonal flow can be characterized 
by ten steps. The critical steps are concerned with the selection of an appropriate adaptive model and its 
application in the composition of an ij\struction?'J prescription. 

In reference to Figure I , the AIM utilizes the following steps in individually guiding the students: 

Step I. The student's learning profile is updated based on immediately prior performance, 
learning time, and associated data from similar learning tasks. 

Step 2. The current task characteristics with their associated behavioral objectives are retrived. 
Most importantly, these task characteristics will related prepositional statements 
concerning the type and kind of learning processes involved. 

Step 3. All available instructional alternatives for the associated Instructional tasks are retrieved. , 
Table 1 presents a list of the type of instructional alternatives which are being considered. 
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Step 4, Any essenUal student characteristic data present in the computer data base are retrieved if 
the data arc likely to be utilized within the instructional decision process. 

Step 5, An iippropriate adaptive decision process based on effectiveness data from the prior 
application of the model for the task and the student is selected. In essence, an 
appropriate adaptive model for the task should be chosen. 

Step 6. An appropriate instructional strategy is derived from the model. This strategy should 
provide for selection among instructional alternatives. 

Step 7, The specific instructional alternatives are identified and checked as to their availability. 

Step 8. An instructional prescription is transmitted to the student. 

Step 9. The instmction proceeds. 

Step 10. The evaluation of the student learning will supply critical data to the updated student 
learning profile and effectiveness data for the AIM'S, 

The flowchart can be successmly applied for the task sequence. As illustrated in Steps 5 and 6, 
the essence of the flowchart concerns the different classes of adaptive models. Each of the models can be 
characterized both in terms of its purpose and its quantitative characteristics. Perhaps a brief review of the 
purposes will give some idea of the essential characteristics of each class or type of mode!/ 

The primary objective of the Orill-and'Practice Models is to increase the accuracy and speed of 
student performance"^bh're^)^titive tasks. The primary objective of the Simple Tutorial Models is the 
acquisition of new conceptual behaviors. These behaviors may concern new definitions, dimension of 
concepts, and relationships among these dimensions. Complex Tutorial Models concern task situations 
involving two or more concepts, simulation representing the concepts, and/or problem-solving applications 
of the concepts. The primary objective of Algorithmic Models is to provide a systematic, efficient 
approximation toward some specific goal. On the other hand. Dynamic Program^ning Models provide for 
solutions that minimize learning time and offer the best utilization of resources. Natural Language Models 
have a primary goal of reproducing the dialogue between two human beings and, therefore, represent all the 
techniques and strategies utilized by a human instructor. Finally, Automaton Models strive for the full 
representation of all mental processes and their changes while training is administered. 

TABLE 1. JNSTRUCTiONAL ALTERNATIVES FOR ADAPTIVE MODELS 



AlterDative Method 


Advantages 


Alternative media presentations 


Provides for the selection of appropriate media types at choice polnH Having two or 
more available media treatments, e.g., film, slide-tape, video tape, text, lecture, com- 
putcr-Lnteractive problem solving, TV, and skill critique. 


Type or interactive instruction 


Provides for selection of a training activity that has the appropriate intetactive rates 
and characteristics, e.g.»PI,CAl, simulation, performance with cquipruent, instructor, 
tutorials, and student-to-sludcnt tutorials. 


Entrance into a learning hierarchy 


For those task sequences that have a generic learning hierarchy, the criteria for each 
entry point in the hierarchy will be identified and students assigned entiy into the 
hierarchy according to their characteristics and current performance. 


Sequence of topla in ietms of 
level cf difficulty 


Provides for matching student performance characteristic with alternative topic 
sequences and associated redundancy levels, whether they are summary, normative, 
or elaborated. 


Student ai>d system control pacing 


Provides for unique studenl-bawjd training time limits for an instructional task (student 
paced) and for system pacing-assignments for computer-uiUractive training activities. 


Amount or practice 
,^ — — ■ — ' 


For each set required of a student, the number of probl^^^^^ will be unkjuely derived 
according to hh pcrformanv? history. 
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Relationships among Adaptive Instructional Models 

Ihe six proposed classes of AlM's are interrelated from two points of view. The first concerns 
commonly shared or unique training variables; and the second, the incluaveness of the tearnijjg and 
resource allocation process. There appear to be five categories of variables that reflect the operational' ' 
characteristics. These are (1) task characteristics, (2) instructional mode, (3) decision processes, (4) student 
characteristics, and (5) instructional resources. The category of task characteristics is highly similar to types 
of learning In that there is a specification of the training requirements from an informational point of view. 
Examples within this category would include items such as verbal Information,, rules, problem sets, 
psychomotor tasks, and complex simulation tasks< The critical variable within-task characteristics concerns 
teaming difficulty. For example, each type of learning will be reflected in a level of performance difficulty 
which will, in turn, influence training decisions such as the availability of prompts or the amount of 
pracljce. The models will vary as to the nature and complexity of the task that they can monitor and adapt. 

Instnictional modes include elements characteristic of a training session, such as the directions for 
learning, the presentation of learning media and materials, type of student interaction (e.^., pacing), and 
feedback. The critical variables in modes of instruction are concerned primarily with media, the type of 
interaction, and the feedback process. 

The models vary as to the number oJ instructional mode variations they monitor and prescribe; 
drill-and-practice varies only with respect to pacing, while alogrlthmic models attempt to prescribe all the 
Variables. 

The decision process is concerned with levels of precision and prediction according to the procedure, 
be it heuristic, algorithmic, or optimization. Another way of viewing the decision processes would include 
the concept of "sufficiency,*' ie,, the degree to which the student's learning data and aptitude 
characteristics are utilized in the decision process. Thus, the more frequently individualized data are utilized 
In the decision process, the more complex and predictive the process becomes* Drill-and-practlce models 
tend to use simple decision rvles, while dynamic programming models employ complete and much more 
complex optimization routines/ 

The fourth dimension concerns student characteristics. These characteristics vary according to stable 
indices such as aptitudes and trait measures, semistable characteristics such as perceptual and learning, 
styles, and task specific characteristics such as latency and anxiety. The models vary considerably according 
to the amount of information they utilize in the process of making decisions. In addition, the models vary 
in their treatment of the variable classes and the collection-update process. 

The last category of variables concerns instmctlonal resources. In essence, a model must construct a 
prescription which furnishes training rescmrces to a given individual for the purpose of achieving a ^ven 
training task. Again, models vary in (he number and quantity of the resources being monitored and 
allocated, whether resources are traditional texts, PI texts, films, computer terminals, trainers, 
simulators, or human instructors. For the different models, there is a range of complexities in the allocation 
of resources. Thus, the five operational characteristics of task structure, instructional mode, decision 
processes, student characteristics, and instructional resources provide the salient diffeiences that aid in 
distinguishing among the models and their potential for improving the training process. 

Research Findings and Recommendations 

It is fully recognized that in the succeeding sections of this report, specific, detailed, and operational 
recommendations arc frequently made as if the research findings fully justify these at present. Clearly, the 
status of the research in the adaptive modeling area on which this report is based is far from being 
informationally or theoretically sufficient to justify each of the recommendations tn alt Aeir operational 
terms. On the other hand, it is precisely these operational recommendations which are being sought. Ways 
to reconcile this paradox are (1) to create the best possible interpretation of the state of knowledge, and (2) 
to provide for future evaluations and revisions. It is anticipated that some of the recommendations madc-in 
the present report wfll have to be revised or dropped in the light of data accumulated in the future 
operation of AIS. For each of the model categories, extensive research Is required to provide complete data, 
interpretation, and guidelines for practice. Nevertheless, since training has to be carried on in tlic present, 
the best availaUe inferences from the existing data base have been made. It should'be recognized that the 
remaining sections of this report do not make continuous cautionary notes regarding the incomplete data 
base in order to avoid unnecessary repetition. 
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ML DRILLAND-PRACTICB MODELS 

Characteristics of the Models 

In general, drill-and-practice may be defined as ihe prcscnlaUon of a series of similar items to which 
the student must respond within fixed time limits. In computer-administered drill-and-praclice, immediate 
analyds of the response and provision for feedback to the student are possible. In the early 1900's, there 
was a great deal of emphasis on drill> in accordance with the "faculties** theoiy which dominated much of 
the educational practice of the day (Suppcs, Jemian, & Briarf», 1968). In a paper summarizing 48; 
unpublished studies, Wilson (1925) concluded that a drill should contain the following attributes In order 
to be effective: 

It should be on the entire process 

It should come frequently in small units 

Each unit should be composed of mixed items 

It should have a time limit 

Examples in a unit drill should be in order of difficulty 
It should include verbal problems 
It should facilitate diagnosis. 

Drill-and-practice 'sequences presuppose the prior introduction of the concept or skill being drilled. 
Strategies have led to the insertion of drilNand-practice ^t three different points in the sequence of 
instruction. These points are (1) immediately after the concept or skill has been introduced, (2) at a time 
when remedial instructional treatment is required, and (3) periodically as a review of the concept or skills 
previously introduced. 

Drill strategies are appropriate for several different types of learning. The performance required of the 
student may involve rote recall such as learning arithmetic tables, spelling words, foreign language 
vocabularies, and symbols for chemical elements. Another application Is to facilitate learning rules or 
algorithms such as those Involved in arithmetic operations, grammar, and the formation of chemical 
compounds. Even high levels of learning such as the abstract reasoning involved In solving verbal analogies 
or troubleshooting can also be handled using drill techniques. 

The two primary goals of drOi-and-practicc scratches are to improve the student*s accuracy and to 
increase his speed of performance. Both of these goals are consistent with the objectives of AIS. During the 
AIM project^ three categories of drUI-and-practice models have been identified as being most useful and 
feasible for the AIS. These categories include pacing, domain-sampling, and domain-exhausting. Pacing 
models are characterized by the utilization of techniques for controlling the presentation rate of drills based 
upon individual history. A specific model for pacing of reading is presented in this report. Much of the 
literature on individualization has speculated on the value of reading at one*s own pace. The model 
discussed utilizes procedures which quite possibly may be more effective than learner self-pacing. 

The other two types of models are more concerned with drill content rathe' than time. The 
domain-sampling model draws items from a largp pooI» whereas the domain-exhausting model utilizes all 
possible Items (content wise). The distinction between domain-sampling versus domain-exhausting drflls is 
significant because the decisions within the two strategies are different for item sequence, level of 
difficulty, tenminalion criteria, and number of items presented. 

Role in AIS 

Several applications for drill-and-practice are evident within the context of AIS, In the Precision 
Measurement Equipment (PME) course, drill-and-practice techniques would be useful for instruction in 
basic mathematics, for teaching definitions of terms, for performing calculations (e^.., calculate voltage, 
current, and resistance in a series DC circuit), for converting units of measurement, and for reading meters 
(depending upon the availability of some sort of visual-display device). In the Weapons Mechanic CoUi«j, 
the procedure could be used for elementaiy topics in electricity. If visual display devices are available, 
drill-and-practice techniques could be used for identifying weapons from photographs or parts of clrcuitty 
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fiom schematics. Applications in Inventory Management include drill in the use of coding schemes and the 
classification of information according to security rules. 

layoff 

The payoffs for drill and practice adaptive models are consistent with major objeclivcs for the AIS 
and are readily stated. At a genera] level, increases in speed and in accuracy of perfonnance in a range of 
tasks varying from simple to complex are two of the possible payoffs. These are gains above the standard 
self-pacing or group-rating instructional strategies, and they increase the viability of AIS goals for a 
cost-effective, individualized system. Furthermore, the models are embedded in strategies for review and 
remediation which are especii^Jly important in a technical training environment. 

Literature Review 

This section summarizes the literature related to computerized drill-and-practice models. 

Orill-and-Practice Implementations 

Stanford Project, In January 1963, the Institute for Mathematical Studies in the Social Sciences at 
Stanford University began a program of research and development in computer-based instruction. The first 
operational instructional program was in elementary mathematical lo^c. During the 1964-196S school year, 
arithmetic drill-and-practice materials were developed and tested. This driU-and-practice program was 
expanded, and it has now been implemented with thousands of students at remote terminals scattered 
around the country (Suppes, Jerman, & Brian, 1968). 

The Stanford curriculum was divided into concept blocks which were further subdivided into lessons 
consisting of 20 items. The blocks were arranged to correspond to the topics in conventional textbooks, 
and the initial instruction on all concept was given by the teacher. For each Item, the student was given 
three opportunities to respond conectly within a 10 second time limit before he was given the answer. The 
next exercise was then presented. Feedback consisted of messages such as *'WRONG," *'TIME IS IIP,** and 
**WRONG, THE ANSWER IS: ^ A student would complete one lesson each day with an average of 5 
minutes for each session at the terminal. Lessons were prepared at each of five levels of difficulty within 
each concept block and organized according to the diagram shown in Figure 2. Those students who scored 
above 79 percent correct on a lesson moved to the next higher level the following day. Students scoring 
between 60 and 79 percent stayed at the same level, and those scoring below 60 percent were given a 
simpler lesson at the next lower level. 

Current projects at Stanford directly attack the problem of adaptation. Adaptive teaching systems 
(Laubsch, 1969) which can control the learning process by itemptesentationsin an optimal sequence have 
been studied. In addition, new research has recently begun on structural variables which reflect 
information-processing attributes of humans when presented wiih a sequence of instructional presentations 
(Loftus, 1970; Atkinson & Juola, 1971; Jerman, 1971). Tliesc attributes are especially relevant to 
sequencing drfll-and-praclicc so as to maximize gams m speed and accuracy. 

Kansas 07>\ A sli^tly different approach to using drill-antl-practicc techniques w^s taken hx ihc 
Kansas City, Missouri public school CAI Laboratory. Most ol ilic lessons were tutorial iw r ^ilure, but 
frequently they contained drill«and-practice sequences. The losions were classified as cidier required or 
elective and were organized into blocks of approximately 12 weeks' duration (40 to 60 niimrcs/week). 
Since no attempt was made to keep students together, there was only a rougli corrcspontk'tii. .* between 
lesson topics and classroom instruction. Each block began with a lesson on using the basic oper;uions with 
whole numbers, fractions, of decimals which wasprmiarily in the drill-and-practice mode. The objectives of 
these initial basic skill lessons were twofold: (I) to screen out those students who had not yet inastcfed 
these prerequisite skills and, consequently, would be unable to succeed in later lessons in the block, and ( 2) 
to act as a refresher for those students who were already close to reaching criterion. Complex pretesting 
strategies were utilized to minimize testing time. Some students were able to finish such les^yons in as litde 
as 20 minutes, while other students required S hours or more. As time passed, the distribution of students 
over lessons within a block became even greater. Wivlc this approach only individualizes instruction by 
adapting to entry behaviors and allowing self-pacinp, it promises great payoff for AIS, particularly in 
courses where entering students vary greatly in experience and ability. 



18 



DAY 

1 2 3 4 5 6 7 8 9 10 



LEVEL 5 



• LEVEL 4 



• LEVEL 3 



• LEVEL 2 



• LEVEL 1 
% 

Fig. 2. Diagram of Branching Structure Followed In Constructing Sets of 
Exercises for Concept Blocks (Suppes, Jernan, & Brian, 1968) • 



The several lessons that were implemented in Kansas City included mechanisms to adapt the drills to 
the individual students on the basis of within-course performance vaiiables such as latency and 
achievement. In a remedial drill on multiplication facts, an algorithm was included to adjust the Um« 
allowed ''^r response input on the basis of recent response latencies and the accuracy of the immediately 
preceding response. Such a pertormance^ontingent pacing condition was intended to improve the student's 
speed of responding without sacrificing accuracy in a task which is basically memorizing the multiplication 
table. In the same drill, a mechanism was included to note the particular stimulus items to which the 
student was responding incorrectly. These items were then presented more fa^quently than woidd have 
occurred by chance (ie.,, if all items were composed by a random number generator). Consequently, the 
student received a greater proportion of items for which it was known he needed practice. At the same 
time, these difficult items (for that particular student) were interspersed with randondy composed items to 

(1) give the student some successful experiences and (2) locate other troublesome items. 

Several adaptive strategies were employed to determine the number of items to be presented in each 
drill. These strategies were typically based on cumulative achievement indices within each drill such as (1) 
continuing to present items until the student gives a specified number of consecutive correct req>onses and 

(2) beginning with a ^ven number of items for a drill and incrementing this index for each Incorrect 
response and reducing it for each correct response until the index either reaches zero or becomes so high 
that remedial instruction is indicated. 

In one lesson, the criterion for passing a drill was dynamically adjusted according to the number of 
times the student had attempted that particular drill. Tlus strategy was employed to avoid trapping a 
student in an instructional sequence which was not effective for him and, concurrently, the proctor was 
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notified that some al tentative instniction should be provided for thisstudent. In Kansas City's curriculum^ 
the feedback consisted of a series of hints or prompts designed to lead the student to the correct answer. It 
the number of inconect responses to an item exceeded the number of hints (usually four or five), the 
student was shown the correct answer and was required to type it in before contLnuing to the next exercise. 

In addition to these two prefects drill-and-practice sequences have been developed for many different 
installations. Most numerous, perhaps because they are easiest to develop, are the drill-and-practice lessons 
in element^ arithmetic skills. Stanford began Its arithmetic drill-and-practice program in the spring of 
1964 (Suppes, Jerman, & Brian, 1968). Other examples of arithmetic drill-and-practice may be found at 
IBM Education Research Kvision, San lose, California (Barnes, 1970; Dean, 1969). Leeds University 
(Woods & Harley, 1971), and Wakulla County, Rorida. (Hansen, Johnson, Durall, Lavin, & McCune, 1971). 

In the language arts area, drill-and-practice has been used for reading at Stanford (Atkinson, 1967), 
WakuDa County, Rorida. (Hansen e j/., 1971) and City University of New York (Schiavone, Rowen, & 
Farrell, 1971). Research in spelling drills has been conducted at Stanford (Fidunan, Keller, & Alkinson^ 
1969). Drill-and practice courses in foreign languages have been developed at State University of New York, 
Stony Brooks (Morrison, Adams, 1969; Adams, Morrison, A Reddy, 1969). 

A rather unusual application of drill-and-practice has been reported by HuUfish at State University 
College, Brockport, New York (HuUfish, 1971). He proposed that drill strategies may be used to teach 
abstract concepts. He described a drill-and-practice pro-am in verbal analogies which explains the logic 
behind each example in an effort to improve the students' abilities to solve them. 

LeanUng Effectiveness 

The learning effecthreness of various strategies associated with individualized learning has been 
investigated along five dimensions relevant to driU and practice models. These dimensions of pacing, learner 
control, massed versus distributed practice, overt correction, and item difficulty are discussed in the 
following subsections. 

Pac&tg, Previously, the capabOity of learning at one*S characteristic rate was generally acknowledged 
to be an important aspect of individualized instruction. However, this review specifies the lack of empirical 
evidence supporting self-paced instruction, and the relative Ineffectiveness of current alternatives to 
self*pacing, and discusses an alternative based on perfomiance-contingent pacing in computer-assisted 
instmction. 

The following review begins with a consideration of self- and fixed-paced instruction. Next to be 
reviewed are those studies which investigated the effects of various externally applied tempos on 
performance in programmed instruction. This review is then followed by a discussion of a method of 
maximizing learning effectiveness and efficiency in computer-based instruction by using pacing. 

Learning with Self and Fixed-Paced Instruction, Feldhusen and Birt (1962) presented cdlege 
students with a 37-frame linear program. Two conditions were employed-one condition permitted students 
to pace themselves, while the other forced the students to proceed^t a predetermined pace. The pace for 
the latter condition was determined by the average time per frame required by subjects v^o worked 
throu^ the program at a self-adopted rate. No statistically significant achievement score differences were 
found between the self- and externally-paced rate employed. 

Carpenter and Greenhill (1963) investigated the effects of self- and fixed-pacing on achievement in a 
15-unit mathematics course. The externally-paced group was presented frames at a rate based on the 
average rate of a pflot group of students who studied the program at their own rates. Under these 
conditions, no differences in final achievement between the self* and extemally^aced groups were 
observed. In contrast to the preceding findings, evidence from a series of studies (Locke & Bryan, 1969) 
consistently indicated a statistically significant performance difference in favor of externally assigned 
(incremented fixed rates) as opposed to self-assigned performance rates. Though incentives appeared to not 
influence performance directly, such incentives were significantly related to performance intentions 
measured prior to actual performance. It may be that incentives persuade an indi^dual to accept assigned 
difficult tasks vrfth a subsequent increase in his performance, wliereas self-selected performance rates do not 
sufficiently challenge the individuates performance capabilities. 
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Kress and Cropper (1964; report NoM) investigated tndividual difr(;r^ 
programmed Instruction, In this Investigation, two 100-frame science programs were administered to two 
groups of eighth graders. Individual difference variables such as IQ, reaillfig comprehension, and entry level 
were investigated. Data Indicated that self-adopted fast paces were related to poorer performance than the 
peiformance of students v^ose self-adopted pace was slow. Most of the variabiHty> however, occurred 
among tow IQ students, suggesting that low IQ students who were pemiitted to pace themselves did not 
benefit from self-pacing opportunities. Also, it should be pointed out that many IQ students worked aJt r 
a slow pace; thus, they were extremely inefficient, even though they performed well. Another important 
observation from the Kress and Gropper study was that students tended to be consistent in the work rate 
which they adopted. The average correlation between work rales on the two instructional programs in this 
study was 0.80. In addition, students were consistent in the number of errors which they made ori different 
programs, as evidenced by a correlation of 0.78 between the number of errors made on the two 
instructional programs. These correlations may indicate that work rate is Idiosyncratic In nature and 
rellective of the student's characteristic reading speed and work habits. 

Since the preceding study suggested that some high- and low-ability students failed to profit frwn 
self-paced rales, the use of such rates in the determination of Tixed rates is dubious. It is reasonable, 
therefore, that Feldhusen and^Birt (1962) and Carpenter and Greenhill (1963) found no differences 
between self- and fixed paced groups. Studies which did not determine fixed paces on the basis of rates 
established during self-pacing are reviewed below. 

Ffeyel (1967) designed a study to compare the effectiveness of self-paced and group-paced instruction 
in teaching a manipulative skill and related cognitive Information. Students receiving the group-paced 
instruction were permitted to progress only after the last person in the group responded to a frame. A 
cognitive and a performance test were administered immediately and at 2 and 6 weeks following 
instruction Heyel observed no differences in performance, students using the group -paced instruction 
required less time to complete the learning task than did the last individual in the self-paced group. 

Gropper and Lumsdaine (1961) compared programmed instruction presented at a fixed pace with a 
lecture version of the same lesson. The programmed instruction material resulted in higher achievement 
than the lecture; however, this difference was due primarily to the high-abOlty students In the programmed 
group. That Is, hi^-IQ subjects showed large performance gains from programmed instruction, whereas 
low-IQ shidents in the programmed InsUuction group did not differ In performance from low-IQ students 
who received the lecture. For the rate employed, this study Indicated that when heterogenous students are 
paced at a fixed rate, high-ability student perfor.nance increases, but low-ability student performance 
remains comparatively poor. A lack of adaptability to individual differences in ability and characteristic 
reading rafe may be the reason that certain fixed-paced rates were found ineffective for low-ability 
students, 

Frye (1963) argues for the use of homogenous groupings. He investigated the effects of group and 
Individual pacing for homogenous and heterogeneous groups on the rate of learning in programmed 
instruction. All students were required to achieve the same criterion. Homogenous groupings in this study 
were based on IQ and predicted algebra ability. Fiye reported the foDowing findings: 

• Students in the group-paced heterogeneous group took significantly longer to complete the 
program than the self-paced heterogeneous group 

• The homogenous group which received self-pacing did not differ from the homogenous group 
which received group-pacing did not differ from the homogenous group wlilch received 
group-pacing in the time required to complete the program 

• The heterogeneous group receiving group-pacing took significantly longer to complete the 
program than did the homogenous group-paced students. 

Thus, only where there is a wide range of abilities represented within a group of students Is there a chance 
that the learning rate will be retarded by external pacing procedures. However, since this study used only 
the learning rate as the dependent variables, the effects of group and Individual pacing for homogenous 
groups on achievement or errors are not known. 
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The results of studies revtewed so far may be summarized as follows: 

• Self-selected rates are often inappropriate for both Viigh- and low-abiiity students; however, 
more definitive research Is needed. 

• Fixed paces based on $e(f-paced rates are Incompatible with high achievement for many 
students 

• Fixed paces not based on rates established under self-paced procedures do not improve the 
effectiveness of the instruction, resulting in only small gains In efficiency 

• Low-abaity students are adversely affected under both self^ and fixed-paced instruction. 

The next section reviews those studies which employ various presentation tempos in an effort to 
optimize leainlng effectiveness and achievement. 

Variations in Tempo. SUverman and Alter (1961) conducted a study which compared two 
fixed paced groups and one self-paced group. One fixed-paced group was paced at a rate slower than 
necessary for most of the students. Finding no differences between any of the groups ted these investigators 
to conclude (page 41) that "pacing the learner will not impair his perfomiance if care is taken to ascertain 
optimal pacing rates," 

In a 1963 companion experiment, Carpernter and Greenhill investigated the effects of various 
presentation tempos on performance in six programmed mathematics units. The tempos chosen were 80, 
90, 100, and 1 10 percent of a base tempo, defined as the mean time required per frame by a pilot group 
who worked through the same materials at a self-adopted rate. Measures of both achievement and attitude 
toward instaiction served as dependent variables. No differences In perfonnance or attitude were reported 
for any of the four pacing tempos. Nonsignificant differences In achievement were also observed In a study 
by Nicholas (1966) which empjoyed rates of 50, 75, and 100 percent of a base tempo. Id a more recent 
study, Blackwetl (1970) founono differences In performance between students taking three (normal, slow, 
fast) versions of a machine-paced sound/filmstrip program. 

The effects of presentation rates ranging from ISO to 350 words/minute on learning from audio, 
visual, and audio-visual modes were investigated by Jester and Travers (1966). Comprehension scores for all 
modes decreased as the presentation rate Increased. Similar results were observed by George (1970) and 
Rosslter (1971) in shidies on learning from compressed speech. 

One of the most comprehensive studies on the Influence of external pacing on learning from 
programmed instruction was conducted by Kress and Cropper (1964, report No, 2). The aims of this study 
were: "determine whether students differing in ability or characteristic (self-paced) work rate experience 
(the effects of external pacing differentially) and to compare the relative effectiveness of fixed- and 
self.paced program instruction," In this study, eighth-grade students were presented a lOO-frame program 
on electricity under either a fixed rale of presentatlo/i or In booWets at a self adopted pace. Three versions 
(slow, medium, fast) of the fixed-paced presentation were prepared. Preliminary self-paced programs were 
administered to determine the students' self-paced or characteristic work rates. Only those students who 
reached an achievement level of 70 percent on the pieliminaty program were retained for the main 
experiment. The observed work rate, then, was one which permitted a relatively high level of achievement. 
The observed results indicated that Increasing the tempo resulted in Increases in average error rate, which 
did not rise above 16 percent for any condition; also, there were no decrements in achievement as tempos 
increased. Kress and Cropper suggest that achievement would be affected adversely if, as a result of 
increasingly fast tempos, the enoi rate became sufficiently high. 

It was further observed (Kress & Cropper, 1964, report No. 2) that when characteristic work rates 
were controlled, hi^-ability students performed better than low-ability students; and as tempos increased, 
the differences between these groups increased as well. Another finding (contrary to Krers & Cropper, 
report No. 1) was that characteristically fast students outperformed characteristically slow students. This 
superiority of characteristically fast students was evidenced by fewer errors and higher achievement scores. 
Interestingly enough (since all students were ^'qualifiers"), students who paced themselves performed 
poorer than students in the slow fixed-pace condition. 



22 



' ^ ^ , ^ ' ^ • 

These results Indicate that, among other things, work rates may be inappropriate under both self* or 
fixed-pace conditions/ That is, self-paced students often adopt a rate which is too fast for accompanying 
high achievement, and, under fixed-pace conditions, students may be required to work at rates which are 
too fast and not compatible with achievement. In addition, increasingly fast tempos appear to impair tlie 
learning of low-ability learners. 

Performnce Q)rJingent Pacing, li h^s 3l^^^ pointed out that Kress and Cropper (1964^ 

report No, 1) found Individual readers to be consistent in the pace they adopt. This consistency suggests 
that forcing subjects to a more nearly optimal rate through an adaptive performance-contingent procedure 
could have beneficiaf effects on later self-paced reading. If near optimal rates could be externally 
established, a reader might tend to maintain the rate under self-paced conditions. 

Brown and James (1972) investigated one strategy for optimizing information presentation rate. A 
self-paced (SP) condition and a performance-contingent (PC) condiition were utilized. Students read 40 
passages presented via cathode-ray tube and responded to three multiple-choice questions following each 
passage. One group received the first 20 passages under a performance-contingent presentation condition 
followed by the next 20 passages under a self-paced condition. Another group received the same two 
conditions, but in the reverse order. In the performance-contingent condition, the presentation rate of the 
passage was manipulated on the basis of the student*s performance on the preceding passage*s questions. 
The following decision rules were employed: 

• If the student answered all three questions correctly, the rate was incremented for the next 
passage 

• If the student answered less than two of the three questions correctly, the rate was 
decremented 

• If the student answered two questions correctly, and if he answered al! three questions 
correctly on the previous passage, the rate remained unchanged for the next passage. 

It is of interest to note that the SP condition following the PC condition resulted in the highest 
retention performance observed and in a reading rate faster than the SP first condition, suggesting a 
carryover from the PC condition ir^to the following SP condition. This carryover effect is also seen as having 
given rise to the interactions between order and pacing and to be consistent with other findings which 
indicate that subjects tend to be consistent in the pace which they adopt (Cropper & Kress, 1965). Slightly 
longer latencies for the PC condition indicate subjects were employing a longer recall interval as would be 
expected given the higher mean presentation rate in this condition. 

The apparent carryover effect and the absence of undersired effects on anxiety or attitude, coupled 
with only slightly lower retention scores in the PC condition, suggest that performance-contingent pacing 
could be a valuable alternative to self- or fixed-paced instruction* 

Learner Control. The research on learner control over sequence has shown thai those students in a 
learner-control condition usually score equivalendy on a posttest with those students taught under an 
author-controlled sequence. It has been reported that a small percentage of students, when given the 
opportunity, sequence materials differently than instructors would sequence them. Stu<^ents in a 
learner-control condition tend to take equivalent or less time to learn the materials than those learning with 
sequences under instructor contrd. 

A variable of some interest is the amount of control the student should have over the instructional 
sequence. 

The sequencing and student-control variables have been investigated by Barnes (1970), Dean (1969), 
Mager (1%!), Mager and Qark (1963), Kapel (1965), Crubb (1968, 1969), and Judd, Bunderson, and 
Bessent(l970). Dean (1969) using an elementary arithmetic task, reported that learner-control subjects h2d 
superior performances and, depending upon grade level, practiced less than students ^ven a fixed linear 
task. Barnes (1970) concluded that learner control of amount of practice could lead to time savings if (1) 
the learner is ready to assume control and understands his option, (2) the material is meanin^ul and 
relevant, and (3) the learner is motivated to learn the material. 
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Mager (1961) was Interested in whether individual students, when ^von the opportunity to control 
their own sequence of Instruction, would generate a sequence sihiilar to that devised by the Instructor, In 
his study* six adult Ss with educational backgrounds varying from a high school graduate to a Ph. D. 
Independently learned some electronics principles from an instructor. Complete control over the content 
and sequence of the curriculum was left to the student, the instructor answered questions and provided 
instruction, examples, and problems only at the student's request. An average of four 65«minute sesdons 
were held for each S. It was found that (1) the learners tended to begin the course with a tqpfc different 
from that usually selected by the instructor, (2) there was some similarity between learners in content 
sequences, and (3) the learner-selected topic sequence was not similar to that appearing in the usual 
eiectromcs curriculum. These flndta^^ suggest that motivation might be enhanced by allowing the learner 
some control over sequence and content; The learner is thus studying at a point in the curriculum v^ere he 
chooses to study, and is perhaps more receptive to the material. 

In another experiment in which learner-control was investigated (Mager & Qark, 1963), students in 
an Industrial training program were provided with a detailed list of terminal objectives. There were then 
told that they could ask anyone in their department for information related to the objectives. A shorter 
training time resulted for then; students, and it was concluded that the students were better prepared than a 
group of students that had graduated from the normal Job-training program. On the basis of this and other 
evidence reported in their paper, the authors concluded that students can learn on their ovm, and that the 
adult student^ at least, can be a good judg* of what he needs to learn and how he should 1^^ 

Kapel (1965) used linear programs In hlstofy to investigate learner control of sequence with 40 
ninth-grade students. One group used the PI text in ihe usual manner, progressing throu^ the program' in a 
linear manner. The experimental group was encouraged td search ahead in the text at any time for 
information. The Ss in the nonsearching group scored higher on a test of Initial learning, but the Ss who 
-were encouraged to search ahead for information were superior on a retention test administered 1 month 
after completion of the program. Apparently, the learner given some amount of control over sequence can 
organize materials in a manner which is more effective for his ovm objectives. Gnibb (1968) has 
hypothesized that the adult learner can structure such a Curriculum vvhich is suited to his own needs. The 
desirability of a learner-controlled course, wherein a map is presented and the student is permitted to 
inquire into different levels of a course, was discussed, llie author suggested that motivation would be 
enhanced by such a procedure because the learner would be In a part of the course because he chose to be 
there. 

Grubb (1969) reported a test of the learner-control hypothesis. In an IBM training program, two 
chapters in elmentary statistics were taught by computer-assisted instruction to 50 adults. In this program, 
SO Ss were each randomly assigned to one of five conditions. Each chapter of the materials could be learhed 
under either a linear sequence or a sequence under the complete control of the students AU permutations of 
learner control and linear format were represented, as well as a condition in which Ss could control both 
between and within-chapter sequence. It was found that complete learner control produced higher mean 
performance than conditions employing lesser degrees of learner cojitrol. Performance was degraded as the 
degree of learner control decreased, except for the condition where learner control was permited witMn 
both chapters, but not between them. 

Learner control of inslnictional sequence, then, is a condition that has produced promising results. 
For those students capable of sequencing materials between chapters, the option is open, and they can take 
advantage of ?he opportunity to jump ahead or back as required. It has been shown (Mager, 1961), that 
students choose different paths throu^ a curriculum. It has also been shown (Mager & Oark, 1 963 ; Gnjbb, 
1969) that students perform well when choosing a sequence. Student control, then, is seen to be an 
important factor In performance, at least for the materials that were investigated. 

According to Judd (1970), any conclusions drawn about the relative effectiveness of the various 
learner-control options must be qualified due to the students* inexperiences with relatively unstmctured 
learning situations. He also suggests that if students are to be ^ven the option of deciding whether or not to 
enter a particular instructional segment, basing the decision In part on the results of a diagnostic pretest, 
they should also be given control options within the instructional segment. Otherwise, there is a tendency 
,for the students that do need the instruction to avdd entering the instructional segments. Judd found that 
students who were given the option of terminating instruction performed at least as well as those who were 
required to reach criterion before exiting from an instructional sequence. He therefore concludes that 
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students are Indeed competent judges of the amount of practice which they require on given topics. On tlic 
other hand, allowing the students to determine which instructional topics to investigate and the order in 
which these topics are taken appears to have had little beneficial eff^t, as compared to the predetermined 
sequence. Judd further elaborates on ihis result by noting that students tend to select topics for further 
study on which they have had some prior success as indicated by a higher pretest score. According to Judd, 
simply introducing the student to the topic appears to greatly increase the probability that the student will 
persist in studying that topic. Therefore, a degree of programmed control which atleast leads the student to 
the topic may be preferable in situations in which the student has relatively little competence. 

Massed versus Distributed Practice. The spacing of drill exercises is another question which has been 
investigated. When the Items for a drill constitute a finite pool, there are many questions regarding the 
optimal order and number of presentations. With the same number of presentations for each item in a 
computerized jpcUlng drfll, distributed practice resulted in better performance than massed practice 
(FishiTian, Keller, & Atkinson, 1969). 

Another issue which received considerable attention was that of the desirability of udng isolated (one 
concept only) versus mixed (several concepts at a time) drills. Repp (1935) recommended that isolated 
drills be employed following the introduction of a new topic or for remedial work, while mbced drills are 
better for maintaiiiing skills. Suppes (1968) concluded that the optimal block size for learning a list of 
simple Items depends upon the relationship between rates of learning and forgetting. The block size should 
be large when learning occurs faster than forgetting and small when the reverse situation is true. 

Overt Correction. Overt correction may be defmed as requiring the student to enter the correct 
response after he is given feedback on an incorrect response. There is some evidence that requiring an adult 
subject to make an overt correction response after reinforcement has no effect on his learning rale nor 
achievement (Burke, Estes, & Hellyer, 1965). In an experiment with young children, howcver^ the learning 
curves for the two groups show a significantly faster rate of learning for the overt correction group 
throughout the entire experiment (Suppes & Ginrf)erg, 1962). 

Item Difficulty and Structural Variables. While drill-and-practice implies a homogeneous set of items, 
there are frequently variations in difHculty within a pool of items. It, therefore, becomes necessary to 
determine the difliculty of items in a ftxcd pool and to predict the difficulty of items to be randornly 
composed. Two methods have been used to determine item-difficulty indices. One is based on an analysis of 
student response behavior on previous administrations> and the other is based on an analysis of structural 
variables or characteristics of the task (Maloney, 1962), For vertical addition problems (Woods & Hartly, 
1971), the behavioral indices were (1) probability of success and (2) rate of work in each column. The 
structural variables which proved to be significant were (1) digit size and (2) number of rows. Cohen, 
Craun, and Johnson (1971) described the difficulty of spelling items, with a tentative list of 23 predictor 
variables. These stmctural analysis variables appear to be quite promising in terms of scaling and in 
sequencing items from easy to difficult 

DriU^ind-Pnctice Models for AIS 

With the preceding research literature and the experiences of educators at both Stanford University 
and the Kansas City Public Schools as background, two drill-and^practice models appear to be vkble for 
AIS. These two models are described in terms ofinput variables, the model's processes, and the nature of 
the output The first model Is concerned with pacing of students, thereby controlling and facilitating the 
rate by which the learner completes the instructional lesson. The second model is more traditional in that 
the decisions concem^e number, sequence, and difficulty of the drill-and-practice items, the nature of 
feedback, and drjli4ermination procedures. 

The Pacing Model, The pacing model used by Brown and James (1972) (Figure 3) was developed on 
an intuitive basis, and, althou^ it seems to be functioning satisfactorily, there are two considerations \^ch 
point out the need for further refinement of the model. 

First, no provision was made within the model to take into account an established characteristic 
reading rate for each individual. If such a pacing procedure were to be embedded in on going instruction, a 
base rale provision would be advisable. Second, the decision slruction of the model needs to be revised such 
that the optimal rate for ea^h learner is identified as rapidly as possible. A rapid identification of the 
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optima] rate will yield gfcater instructional effectiveness and efficiency. Fuithermore, after the optimal rate 
is located, the decision structure should stabilize the presentation rate. More sensitive adjustments must be 
included in the model to ensure rate stability so that an individual's optimal rate will be approximated to" 
the greatest possible extent. 

hipuf. The suggested pacing model begins with a detemunation of the learner^s self-paced, or 
characteristic rate of reading in terms of the number of words read per minute. The accuracy with which 
the student reads a given passage, in terms of both comprehension and word for word accuracy, will also be 
measured and retained by the computer system. These measures may be coiiMvls^ied as state variables. The 
trait characteristics to be measured may include reading style, reading level, verbal aptitude, and 
vocabulary* Measures which already exist in computer fUes will also be utilized. These measures include 
AQE scores, background meastires such as SES, or motivation levels. 

Processes. The raie of reading which is measured at the outset of instruction will be utilized to begin 
the search for 8n optimal pacing level together with the input variables listed previously. On the basis of the 
learner's performance in the task, as well as on trait measures, the presentation rate will manipulated; some 
rates wSI be increased, while others will be decreased in an effort to find the optimal rate. The outcomes of 
the pacing model, then, are reading rates which are more optimal than rates in a self-paced mode. This more 
optimal reading rate means that the performance has been maximized without creating detrimental effects 
on efficiency. 

Tlie pacing model presented here may have applicabiHty for a wide range of Instructional tasks. 
Essentially, when one or more paragraphs of written material are presented to a student, his reading ability 
becomes a factor in determining his performance. The pacing model would not be limited to one adaptive 
instructional model, but would be utilized whenever appropriate. An additional feature of the model is that 
the students* reading rate will be continually monitored both for adaptive use within the model itself and 
for use in other models as welL 

Output, The pacing model has as its goal tlie facilitation of reading speed without the sacrifice of 
reading accuracy* The model will facilitate the attainment of this goal by providing output in the form of 
reading rate data which can be of use to instructors and as input to other adaptive models. From the 
students' perspective, the model is expected to increase reading rale. 

Traditional Drtll-and-Practice Models 

The variables related to dritl-and-piactlce instruction are divided into two classes, namely, task 
variables which relate to the nature of the instruction; and student variables, which describe individual 
student characteristics. Both task and student variables are used as input to the driJl-and-practice adaptation 
procedure* Performance variables are also utilized within the drill^and-practice lessons to facilitate the 
adaptation process. The following paragraphs describe the input variables which will be used, the decision 
processes which wQl be employed, and the output that can be expected from the traditional drill and 
practice model. 

Input, The input variables are categorized as student, task, and instructional variables. A student 
variable, such as reading level would be cmcial for determining the means of communicating directions and 
the amount of text in the presentation of items. Aptitude and IQ will be used to predict the amount of 
practice necessary and the optimum rale ofincreasing item difficulty. Diagnostic information regarding the 
specific task not only answers the question of whether or not the student needs the drill, but also aids in 
specifying the initial range of item difficulty. Personality traits such as perseverance, and indices such as 
learning style v«ll be used to determine the degree of learner control of instructional jr^rameters. As a 
specific adaptive system is designed, other potentially profitable student variables may be identified. 

The identiCicalion of task variables depends on an analysis of the task characteristics* However, some 
general variables may be specified such as item difficulty. Other task variables include the size of the item 
pool and the task requirements affecting the establishment of performance criteria. Task ch^^i^cterislics will 
also influence the choice of a terminal device since some tasks require audio or visual pTCseni>> ^ori 
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While student and task characteristics remain fairly static throughout a dril!. instructional variables ate 
generated by the drill itself and, consequently, are constantly being updated. They may be thought of as 
slate indicators. Included in this category are items such as cumulative and recent indices of performance, 
response latency, item difficulty, location in Item pod. and number of oroblems presented^The state of 
these Indices for the n*'^ item Is used as Input for decisions regarding the n^'* problem. 

Process, There are many opportunities for adaptive decisions within a drOl-and-practice sequence. 
These decisions include the follovnng: 

• How many items should be presented? 

• How should the items be sequenced? 

• What is the difficulty of each item? 

• How much time should a student be allowed for ca<;h response? 

• What kind of feedback should be given? 

• When should a student exit from a drill-and-practice sequence? 

• What are some criteria for predicting mastery? 

One may categorize drill-and practice models as one of two distinct types on the basis of the domain 
of items involved. The term '*domain-sampling*' model will be used to refer to drill sequences in which the 
Hems are randomly generated or drawn from a very lar|e pool of homogeneous items. Drills in arithmetic 
would be included in this category. Chemical symbol drflls, however, involve a finite list of elements for 
which the student will receive practice on every item in the pool. Drflls of this nature will be categorited 
under the "domain-exhausting** model. The selection of the appropriate model 5s dependent upon the task 
involved. The distinction between domain-sampling and domain-exhausting drills is necessary, because the 
decisions inherent in these two models are different with respect to the number of items presented, item 
sequence^ detemiining item difficulty, and criteria for terminating the drilL 

For the domain-sampling model (Figure 4), the number of problems presented could be fixed for all 
students based on some heuristic decision, or predetermined for each student based on an algorithmic 
procedure using input variables to predict the optimum number of items for that student. Another 
alternative would be to dynamically adapt the number of items presented on the ba^s of the student*& 
performance withid the drfll. For this latter alternative, it would seem feasible to employ a procedure such 
as Wald's Sequential Probability Ratio (1950) for predicting mastery at which point the drill may be 
discontinued. Another possibility would be to allow the learner to control the number of times an Item is 
presented. It is possible that one could predict on the basis of input variables (learning style and prior 
performance) whether it would be better to specify the system control or specify the learner control 
(Barnes, 1970). 

Since items in a domain-sampling model are frequently randomly generated or otherwise composed 
according to a specified format, it is not likely that a difficulty index determined on the basis of previous 
administrations will be known for each item. Consequently, item difficulty indices must be based on an 
analysis of stmctural variables ^uch as the number of componei\ts, difficulty of each component, and 
complexity of relationships among components. 

Several options are available regarding (he sequence of items. The simplest drill would pteseni a set of 
homogeneous items without regard to item difficulty- Another alternative Is to order the items from ea^ to 
hard with the rate of \ncte^$is\g dlificviity either fixed or predetermined for each sfudeni on the basis of 
Input variables (ability and aptitude). Also, a difficulty range could be specified for each person on the basis 
of 6lSii;no%\K information and student characterisiics. In any event, any deliberate increment in item 
difficulty should be performance contingent. 

For (he domain-exhausting model (Figure 5), the number of items io 6c presented depends on the 
total number of items in the pool. The relevant question for this model is to ask how many tirnes each item 
should be presented. Should a subset of items be identified as an ''active^' pool and, if so, what Is the 
optimum size? If the optimum size varies from student to student, what variables are needed to predict it? 
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According to Suppes (1964), the optimum block size depends upon the relationship between rates of 
learning and forgetting. When learning occurs faster than forgetting, It is possible to show that block size 
should be as large as possible. On the other hand, when learning is slower than forgetting, the block size 
should be small. Indices of learning and forgetting rates would, therefore, be desirable input variables for 
thb decision process. 

With regard to item sequence, there is evidence that practice in specific items should be distributed 
throughout the drill rather than occur in close succession (Fishman et al., 1967). Thus, once an item has 
been mastered and, consequently, deleted from the active item pool, it shoud reappear occasionally for 
review purposes. The criteria for deleting an item from the active pool and the amount of review needed 
must be empirically determined. As with the domain^sampling model, it may be more efficient to give the 
student control over the amount of practtee and allow him to request review items. Some early research in 
driU-and-practJce (Reppc 193S) emphasized the benefits of isolated drill (a homogeneous set of items for 
one specific task) following the introduction of a new topte or for remedial work. However, a mixed drill; 
Le, , one containing several types of items» is best for maintaining skills. 

The difficulty of items may be based on structural variables as in the domain-sampling model. In a 
domain-exhausting model, the items are predetermined and not composed within the drfll. Thus, indices of 
item difficulty could also be predetermined on the basis of preWous administrations and used as input 
parameters to the drill. The ultimate adaptive model would dynamically update this Index each time a 
student responds to the Item. Item difficulty is also the basis for selecting the appropriate diill-and-practice 
model, te., domain-sampling or domain*exhausting. If the items for the domain-sampling model are to be 
randomly composed, then item forms must be specified. If the items are to be sampled from a larger pool. 
In the case of the domain-exhausting model, the tojial list of items must be provided. The 
leaming-and-forgetting curves for this type of task shi>u1d be indicated to determine the optimal block size 
of the active pool. 

If item difficulty is to be manipulated within the models then the selection of appropriate structural 
variables for determining difficulty indices is, of course, dependent upon characteristics of the task. For 
vertical addition, these mi^t include digit size and the number of rows (Woods & Hartly, 1971). For 
spelling words, length and troublesome letter combinations (e.^.» ie-ei) are among the structural variables 
influencing the difficulty of an item (Cohen, et aL 1971). A total of 24 structural variables for logical 
problems has been identified (Maloney, 1972). These range from number of characters to depth of 
embedding. From these few examples> it is evidenced that, whfle there are some common factors; e^*, size 
of problem and difficulty of specific elements, the identification of specific structural variables wfli be 
unique to each task with some expectation of carryover to similar tasks within a class. 

Decisions regarding individual frame characteristics are the same for both the domain-sampling and 
the domain-exhausting model, Firsti a choice must be made among alternative media such as teletype. CRT, 
image l>rojec tors, and audio devices. The nature of the task may restrict this selection some. If there is still a 
choice, it could be based on the input variables describing student characteristics. 

there are also many options for deierming the time to allow for responding to each frame. These 
range from no limit at all to a time limit dynamteally adjusted to challenge each student, /.e, pacing. The 
time limit for a particular task may be fixed for all students or predetermined for each student on the basis 
of Input variables. If speed is a critical element of the task criterion, then perhaps it could be specified as a 
goal, and feedback on re^onse latency could be provided to inform the student of his progress toward this 
goal. 

Another frame diaracteristic is the means by v^ich the student will respond. For multiple chofce 
items, he may respond via light pen or typing a character from the keyboard. In adition to typing a word or 
phrase for a constnicted response question, one can imagine items in which the student indicates parts in a 
schematic with a light pen. 

With regard to feedback, it has already been mentioned that respo/ise latency would be useful 
information to give the students In special situations. The more common varieties of feedback inform the 
student of the quality of his response. In addition to telling the student if his response is correct* It is 
sometimes possible to provide diagnostic feedback indicating the nature of any errors. A conunonstiategy 
is to provide prompts or hints when a student makes a mistake and require him to answer again (overt 
correction), 

JC 32 



The criterion of acceptable performance is directly related to the nature of the task. For some tasks, 
speed is a crucial element and assumes a more dominant role in determining the student's success or failure 
for a particular item. In such a situation, the student may be given a speed goal and updated information on 
his response latency. When he consistently responds correctly within the item limit specified by the goal, 
then he has met the criterion. The items In such a situation are usually of very low difficulty. Most tasks 
place more emphasis on accuracy and would, therefore, specify a criterion in terms of proportion correct. 
For some tasks, however, the consistency of correctly responding would be a better indicator of mastery. 
This would include concept identification and discrimination tasks in which the student will always answer 
conectly once he has learned the concept. The criterion of acceptable performance must, therefore^ be 
specified for each task on the basis of the goal for the task and the evidence needed to judge if the student 
has reached the goal. 

Output. The essential to be determined is the number of items to present to the student during 
drill-and-practice. Since different students require a varying amount of practice, it would seem most 
efficient to attempt to adapt the amount of practice to the individual. Thus, students are not wasting time 
with unnecessary practice and, at the same lime, they are receiving enough practice to achieve the criterion. 
Several strategies Will be employed to determine the number of items the student should be presented In 
the drill-and-practice sequence. 

• The simplest model prescribes a fixed number of items for every student. At the end of the 
sequence, the percent conect is determined to predict mastery or nonmasteiy. 

• A second method predicts mastery when a student has given a specified number of consecutive 
correct answers. Such a model would be appropriate for concept4eaming in which the student 
will always answer correctly once he has mastered the given concept. According to Gagne 
(personal communication), one correct answer is sufficient to indicate that the student has 
mastered a rule. However, he also specified that students should be given additional practice in 
later review sessions. 

• A more elegant decision mechanism employs statistical techniques sudijj JVald*s Sequential 
Probability Ratio (1950) test, which uses students' performances pn it^m^f^predict mastery. pr 
nonmastery. The number of items presented depends upon the student's performance on each 
item. ' 

• An analysis of student response latencies will also be useful for predicting masteiy. 
Theoretically, the Initial latencies WJ?U?d be relatively high with the curve decreaslr\g until Jt 
reaches some asymptotic level at which masteiy can be predicted. This model, however, must 
either make the assumption that all items are of equivalent difficulty or include In the decision 
process some means of accounting for variations In item duTiculty . 

• Another means of deciding when to terminate a drill may be to allow the student to judge if he 
has reached mastery. In other words, this model would give the learner control of the amount 
of practice for a drill sequence. 

There is no evidence to show that one of these methods would be more efficient than another. One 
must always be aware of the tendency to be swayed by the more elegant adaptive mechanisms when. In 
fact, a less intricate strategy might be jusi as efficient. 

Recomniendations 

• From the preceding discussion, it should be evident that there are many decision points within 
a drill-and^practice model. Currently, most of these decisions will need both conceptual and 
empirical exploration. It is recommended that these variations be incorporated In the AIS 
research plan. 

• On the basis of the .literature review, it is recommended that two simulations of adaptive 
drill-and-practlce models be developed. The first would involve the performance 
contingent-pacing paradigm and the second would involve an adaptive decision-making 
mechanism to terminate the drill when it is possible to predict mastery or nonmasteiy. 
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• For each review session, a drill-and-practice model should be utilized to formulate the problem 
list, organize the sequence, and provide optimal allocation of practice per problem type. AIL 
instructional decisions should be based on individually determined parameters. 

More specifically for the AIS project, the drill-and-practice model should be utilized in the courses as 
follows: 

• For Inventory Management, the focus should be on coding/index schemes and classification 
structures as in security rules; 

• For Precision Measurement Equipment » the major emphasis should be on technical and 
conceptual aspec is of the course ; and 

• For Weapons Mechanics, the focus should be on conceptual factors of electricity and 
image/photo requirements of specific weapons. 



IV/ SIMPLE CONCEPT ACXJUISmON MODELS 

Chaiacteristics of Adaptive Concept Acquisitioo Models 

Adaptive concept acquisition (ACA) models are represented by instructional paradigms designed 
according to decision processes that adjust instructional variables to individual differences and differential 
learning performance. For the adaptive concept acquisition models, the basic variations proposed are of two 
functional classes-pretask and within-task variables. Pretaslc variables are composed of individual difference 
and task variables, such as abQity and problem difficulty. These variables serve to set limits on the 
instructional alternatives available, and the media to be used for instruction. In the second class, within-task 
variables provide for the manipulating of such alternatives as the number of examples^ the degree of 
prompting, and the nature of the feedback/correctional process based on individual criteria. Thus, the 
fiexibility of the ACA models are distuigulshed by the varying levels of adaptability. In the AIS context^ 
this means that the model can be modified to reflect the diversity of concepts being taught. This is an 
important consideration in that the complexities of the targeted concepts should determine the degree of 
adaptability to individual difTerences. 

Role in AIS " 

The adaptive concept acquisition modeh will probably play a fundamental pervasive role within AIS. 
While the current ATC instructional paradigm consists of lecture, small group demonstrations, individual 
practice, and criterion performance evaluation, the individualized concept acquisition instruction may 
replace lecture and small group demonstrations. As each lesson presents new concepts or reviews and 
combines previously introduced concepts, the individualized media assigrunent, examples, prompts, and 
feedback should facflltate the adaptive process. For the purpose of this report, the primary emphasis will be 
given to the optimal selection of the type and number of examples during concept acquisition. 

Payoff of the ACA Models 

As proposed, the ACA models may become an integral part of the computer-based and conventional 
media approaches to concept presentation and review. Given this high frequency of utOization, the models 
should provide for significant savings In training t^me and improved concept retention. As operational 
features, the following benefits of the application of the ACA models are envisioned: 

• The pretask variables of ACA models are adaptable to individual student trait characteristics. 
Premeasured conditions would assist in the assignment of students to appropriate entiy points 
within the instructional tasks. Such decisions would provide for residual savings in training costs 
by allowing hi^-aptitude students to finiA courses more rapidly, or to receive enrichment 
training. Individualized assignment of low-aptitude persons to appropriate Instruction has been 
shown to increase efficiency. 
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• The witliin-lask variables are designed to select instructional materials based on a student's state 
characieristics. In a concentrated learning environment, individual performances fluctuate so 
that premeasures do not always indicate accurate assessments of current capabilities. These 
within task variables make the presentation self-modifying in that it is continuously being 
adapted to the student's current response pattern and slate levels. 

• Since the ACA model will be computer-based, each student will have Immediate access to 
adaptive instructional materials. 

• Instructional theory concerning media, feedback, knowledge of results, sequencing, role of 
examples, and type of display can be designed into the adaptive individualized packages. 

• The utilization of ACA models should improve cost-effectiveness by providing a more precise 
prediction of the necessary media and materials overlap than is currently available. 

literature Review for Conceptual Acqukitlon ModeKs) 

Instruction is a process of manipulating the environment tp produce a desired change In a student*s 
behavior. The goal of AIS is to implement an instructional system that will take into account individual 
differences so as to increase the effectiveness and efficiency of student learning. Early attempts to solve the 
problem of individual differences have been suggested and developed with varying degrees of success/ On^ 
widely used practice was grouping or tracking of students by grades, or by scores on ability tests as an 
attempt to take into account individual differences. This homogeneous grouping had little effect because 
the groups seldom received different kindsof instruction. Air Force training incorporated Skinner*s ( 1958) 
linear programmed instruction, which allowed students to progress at their own rates. This procedure 
emphasized that individuals do function at different learning rates; however, the material itself was not 
individualized since all students received the same instructional sequence. The Influx of techndogy 
infiuenced Crowder's (1959) procedures of Intrinsic programming with provisions for branching able 
students through the same material more rapidly than slower students, who received remedial frames 
whenever a question was missed. This type of programmed instruction was not wjdely used in Air Force 
instructional situations, or in any other institution, because of the difficult developmental task which 
required review sections for each alternative answer. 

There are two basic procedures for designing concept acquisition instruction, which would have 
adaptWe capabilities extending from the above assumptions. The first involves the use of premeasure(s) 
(such multiple variables as aptitudes, personality variables, and anxiety) for diagnosing the student's 
behavior and then prescribing a specific learning task designed to adapt to these individual differences. The 
second applies intermediate evaluations of the $tudent*s progress within the instructional sequence and 
assigns adaptive segments to correct errors in acquisition. 

Pretask Adaptatk)n 

Cronbach (1967) discussed the applkability of adaptive Instructloj^ to student differences by^ 
suggesting that, If development in a wide range of persons was to be facilitated, a wide range of 
environments suited to the optimal development of each individual must be offered. In terms of AIS this 
would mean having instmctional units covering content available in different formats or sequences which 
. r ^ differences among students. For example, Cronbach's model might prescribe f 

?W Trifi «'^d,'«^dia for a student of certain characteristics, whde another student of differing 

characteristics would receive an entirely different mode of Instruction. The advantage of the ACA model 

^I'h ' m'T^"'''"^'^^. P'"^^"^^ ^^"'^ O^^^aity of selecting decSon co^^^^^^ 

whfch would change according to concept content. vunuuiuns 

.A^Jl.^ltl ^^"^5^ prescribing optimal instructional strategies, Cronbach (1967) 

advocates that an extensive research program be conducted to identify those aptitudes which intereact 
rnaxmially wiUi instiucttonal treatments. This body of research has become known as aptL MreaS 

assignments can be made . rom empirically determined measures existing prior to the onset of instruction 
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Recent research studies (TaJlmadge, Schearer, & Grecnbcrg» 1968; Cronbach & Snow's review, 1969; 
Dunham and Bunderson, 1969; P.F. Merrill, 1970) have Investigated the assumption to determine if 
premeasured individual aptitudes interact with instructional treatment, These studies indicate that 
disordinal interactions of ATI's have an elusive nature. Bunderson and Dunham (1970), in the final report 
of a 3-year research project on cognitive abilities and teaming, challenged the ATI concept as a viable 
predictive procedure in "real world*' Instructional contexts. The reasons for their skepticism can be 
summarized as: (I) the rarity of useful disordinal interactions; (2) disordinal interactions are not 
sufficiently robust after minor changes in the task or population; (3) tlie benefit from disordinal 
in|teractions may be less tlaon iliat attainable throu^ revision of a single optimal treatment. In this report, 
Bufiderson and Dunham (1970) suggest that, instead of seeking disordinal interactions in order to assign 
indh^lduals to different macro-treatments, ATI's be used to revise the optimal treatment to reduce the 
learning burden of slow-aptitude individual! After the effectiveness of the single best treatment has been 
maximized using a systematic approach to instructional design (Bunderson, 1970; Tennyson and BoutwelJ» 
1971). macro-treatment variables can be applied adaptlvely In the instructional program rather thaft 
produce entirely different alternative treatments^ For developmental concerns of the AIS, this would mean 
designing an opHmal concept acqu!$itjon instructional program, using the most efficient sequence, the most . 
appropriate media for display, and the most effective Instructional examples. Adaption within the program 
would then occur when students deviated from the (^tlmal program. 

Withln-Taak Adaptation 

The second procedure proposes adapting instmctional strategy according to a student's behavior in 
the learning program, and to other current state characteristics. The within-task adaptation procedure can 
be contrasted to Cronbach*$ approach in that individuals are not assigned to different macro treatments, 
nor are measures obtal»^ed prior to the entry of the individual into the Instructional task employed. On the 
other hand, the withln-task procedure differs from Crowder's approach in that Crowder utilize^ only the last 
re^onse made by the student in reaching an instruction decision* The within-task adaptive strategy would 
make Instiuctioral decisions bcsed on an updated history of the student's behavior during a segment of the 
concept-learning tasks. Furthermore, the reliability of a pattern of responses compared to a single response 
should increase the validity of such adaptive decisions. In AIS, decisions on media and mode of Instruction 
for particular units would depend on what is the most appropriate method of presentation. When student*s 
deviate from the optimal sequence, they can be assigned corrective instruction. The within-task method of 
adaptation is such that remedial "hole patching" (Cronbach, 1967) is avoided on the basis of instructional 
theory and can be validated empirically. 

There is research evidence (O'Nefl, Hansen, ASpielberger, 1969; Leherissey,0*Neil,& Hansen, 1971; 
Tennyson & Woolley, 1971; P. F. Merrill & Towle, 1971) that trait or state variables measured prior toa 
learning task are not as effective in predicting student performance as state variables measured during the 
actual learning of the task. These findings suggest that it would be possible to Include such measures during 
the task to adapt Instructional sequencing for those students at the extremes on these measures. 

The within-task adaptation model is based on three basic assumptions: (1) there aic n limited number 
of different kinds of behavior or types of learning (Gagne. 1970; M. D. Merrill, 1971); (2) there is an 
optimal group instmctional strategy or paradigm based on the conditions of learning for earfi behavior 
level; and (3) individual performance can be optimized by making adaptations to the group instmctional 
paradigm according to individual response patterns. 

Concept AcqulsH fen 

Mechner (1965) defined concept acquisition as the process of generalizing within a class and 
discriminating between classes. For example, in the Weapons Mechante AJS course, students would have to 
identify certain types of wiring systems, and at the same time, discriminate between the systems* To teach 
this skill, Markle and Tiemann (1969) and M. D. Merrflt (1971) postulated that concept acquisitions would 
result if examples during instniction differed in the irrelevant attributes associated with each; that is, each 
kind of wire should be system presented in- many different colors, thkknesses, structures, etc. Such 
presentation promotes generalization within the class. Discrimination between classes Is facilitated by 
presenting nonexamples which have Irrelevant attributes resembing those with given examples; for the 
wiring illustration, the various wiring systems would be nonexamples for the one system under mstructlon. 
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In testing for concept acquisition; it is vital that the items on the test must be new, not used in prior 
instmction. A good set of items must have a number of other characteristics. In order to test for 
generalization across the total range of examples included in the concept, test Items must cover the range 
^ecifled by a thorough analysis of the concept. The number of examples the student can correctly classify 
Is less important than the range of examples to which he can generalize. Discrimination of nonexamples can 
also be tested when the analysis of the concept has identified the key relevant attribute^, 

Tennyson, WooUey, and Menill (1972) designed an opthnal gioup instructional strategy for teaching 
concepts based on the theoretical work of Markle and Tiemann{ 1969, 1970), Merrill{l971), and Wodley 
and Tennyson (1972). The concept that Tennyson et a/., chose to teach was the metrical concept, "trochaic 
meter," as exemplified in poetiy selections. As a preliminary estimate of range* they asked students 
unfamiliar with poetry to classify a large number of examples and nonexamples of the concept on the basis 
of a gWen definition. Some obvious examples were recognized by almost all subjects and were, therefore, 
termed high-probability examples. Some nonexamples were equally obvious and were termed 
hi^-probabflity nonexamples. Examples which were difficult to recognize were termed low-probabflity 
examples; subtle discriminations \^ich could not easily be made on the basis of the given definition 
produced low-probability nonexamples. Thus, both range of examples and fine discrimination of 
nonexamples were defined in their study on the basis of ratings by representative subjects rather than on a 
prior analysis of the concept. 

Tennyson, WooUey, and Merrill (1972) hypothesized that different combinations of these high- and 
low-probability examples and nonexamples would produce predictable enors in concept acquisition. Markle 
and Tiemann (1970) had proposed that restricting the range of examples would cause a student to 
undergeneralize, that 1$, to accept on a (est only the sanw limited range provided In instruction. Tennyson e/ 
al., produced precisely this effect by giving students instruction v^ich included the definition, only 
high-probability examples, and the subtle discriminations taught by the low-probabilily nonexamples. They 
also proposed that poor selection of nonexamples, in conjunction with a broad range of examples, would 
cause students to overgeneralize and to accept nonexamples as members of the class on a test. This effect 
was produced by providing instmction including the definition and full range of hl^- and low-probability 
examples but only very hi^ probabQity nonexamples. In other words, no difficult discriminations were 
taught, and, on the test, these students did not succeed in making such fine discriminations. 

In their study, they also demonstrated the effect of a particular kind of limitation on the range of 
examples, in which one salient but irrelevant attribute is always present. The attribute used was Victorian 
origin of the selections. All examples of trochaic meter given students in this treatment were dated in the 
Victorian period, while nonexamples were selected from earlier or later periods. Despite the definition 
directing attention to the meter of the examples 9r the critical attribute, students showed a misconception/ 
on the test, that is, they generalized correctly only io examples of trochaic meter written in the Victorian 
period. They rejected true examples from other stylistic eras and accepted some Victorian nonexamples. 

Tennyson, WooUey, and MerriU's data support the position that the selecti<m of both examples and 
nonexamples is an important item In effective concept teaching. A wide range of examples prevents 
undergeneralizalion, while a good selection pf nonexamples prevents overgenerallzatlon. In AIS 
developmental projects, (he Tennyson^r^r/., model has application to the actual design of the ACA 
instniclional materials. TTie system provides a method for selecting instances and sequencing them to an 
optimal taskiThe component variables are uniquely adaptable to individual characteristics. Thus, they have 
the capabilides for wlthin-task adaptation, for example, if a student is committing a certain kind of 
classificatfon error on an Intermediate evaluation, the type and degree of examples and nonexamples can be 
adjusted to correct the enor. The model also allows for designing a multiple^ntry program based on 
pretask measures. Students with poor reading abSity, for example, would enter the task with easier 
hl^-probabflity instances than someone with good reading abflity. 

Applications of CAI Concept Acquisition 

A numberlof CAI applications using various types ofadaptatlon have been implemented with a good 
degree of success. A number of them are reviewed in this section. The first applicati<Mi uses the Tennyson e/ 
a/., concept acquisition paradigm in individualized CAI programs. The other projects oft^er a summarized 
review of the feasibility of computer^ontroHed adaptive instructional models. The review also introduces 
other ad^tive characteristics that can be included in the ACA model. 
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Brig}tam Young University TICCrT. TICCIT is an instructional development project which is 
undergoing continued deveJopment a( Brigham Young University. The project consists of developing CAI 
courses in the areas of mathematics, English composition, and both remedial English and mathematics. The 
instructional materials are being developed according to the research paradigms developed in the Tennyson. 
Woolley, and Merrill investigation. 

The objectives of the project encompass both cognitive and affective outcomes. Ttxese are, fltst, that 
students will achieve mastery at any of three levels of sophistica!ion(the students contract individually for ' 
a desired level). The second objective is that students will develop a more positive attitude toward the 
subject matter. Third, students will develop improved strategies for learning. Finally, students wiH develop a 
sense of responsibility for their own learning. It is planned to achieve these objectives by three main 
approaches: (1) control structures, which provide for learner control, (2) the construction of a high-level 
advisor program," and (3) the use of a variety of motivational techniques. 

The control structures provide for full learner control of instruction, which is expected to encourage 
approach responses to the subject matter, enhance learning strategies, and develop the student's 
responsbility for his own learning. Within each unit of a course, the student will be able to select from a 
"menu," giving aspects of the unit he wants to see and the sequence in which he wants (o see them. A 
typical menu includes the following: 

• The objectives of the unit which give an indication of the structure of the lesson 

• Review tips, which discuss the prerequisites for the lesson and provide brief review material 

• A "so what?" section, which tetls the student why he should bother to learn the lesson 

• Amini-lesson, which is a quick survey of the entire lesson 

• A dennition section in which the major concepts involved in the lesson ire defined 

• The instnictional section, which provides structured instruction (the content of the lesson) and 
teaches the algorithms for performing the skills involved in the objectives. U is within this 
section that the main features of the teaching paradigm described in this paper are employed. 

• Themastery test, which diagnoses the student** degree of success or failure. 

The advisor program used within the project may be called by the student at any time for advice 
regarding the best strategy lo adopt for a particular lesson. The student can also request a status report to 
see what lessons he has completed. He may further ask if, judging by his achievement profae, he could take 
additional options which will enhance his grade. The advisor program is also used to ''advertise** certain 
fun options" which are included in the course for motivational purposes. 

Motivational techniques being developed in the project include the "so what?" section in each lesson, 
the fun options, and a point system used to earn these fun options. The fun options are made available to 
the student contingent upon his success on the mastery test of a given lesson. They include such items as 
"more on the topic," a comput^iionBl plotting program (for the math course), games, short films, and 
tidbits such as historical vignettes, anecdotes, etc. related to the lesson. 

In SMmm^ry then, the control structures permit the student to adapt instruction to his own particular 
• needs, while the advisor program provides the student with the pertinent information needed for him to 
adapt the instruction successfully. The motivalional techniques, finally, provide the incentives which will 
help him oplimi^e his learning. Specific adaptation within each lesson is also provided in the instruction 
part of the unit, where the concept-teaching paradigm elaborated in this report is employed. 

Stanford Project. At Stanford University^ Friend and Atkinson (1971) developed a course entitled 
"Introduction to Programming" for use by NASA personnel. The course consists of a set of 50 lessons, each 
about 1 hour in duration. The course teaches programming concepts through a %mp\e computer language 
called AID, and makes a provision for student practice throu^ a number of exercises designed to help the 
student solve simple programming problems. These lessons are tutorial in nature, that is. no previous 
knowledge of computers or programming is necessary. The branching logic used in problems permits five 
discriminations between student responses, thus greaUy refming the remediation process. That is, not only 
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may a response be diagnosed as "correct" or "incorrect/* but degrees of conei;tness can be established for a 
given problem. This response analysis is made by means of twelve basic analysis routines which can return 
different values of correctness, thereby refining tlie remedial feedback to the student. 

In addition to this Implict branching, the student can also initiate branching by requesting additional 
information throu^ the use of HINT and TELL commands. When the student encounters some difficulty 
in solving a problem, he may thus request a hint which will set him on the right track, Currently, iv/o hints 
are available for most problems and as many as six are provided for partteularly difficult problems. The 
TELL command, on the other hand, will cause the system to print Uie correct answer to a problem on 
whidi the student is really stuck. 

A further optimization scheme is available. In this plan, the response of a student determine the 
number of problems that constitute a lesson. In a given lesson for example, one student may do 30 
problems, while another may do as many as 70. Specifically, the number of problems presented is governed 
by a simple heuristic which will cause the student to bypass all subproblems related to a top-level problem 
which is correctly a^iswered. Through these decision processes and the strong provision for student control, 
the system allows the better prepared and more able students to keep up a fast pace while at the same time 
enabling the less able students to progress at their own rate, thus, making use of additional and remedial 
information. 

Fort Monmouth Army Project , A major tutorial CAI application was developed by the Army (GruntI 
& Longo, 1 971) at the Fort Monmouth Anmy Signal Schoo!, The instructionaj model utilized In the course 
specifies 4 sLngle-track teaching sequence. However, individualization is accomplished by dWiding the 
trainees into high-, middle-, and low-aptitude categories. Each category is then treated differently. Each 
lesson includes a pretest, but only the high- and middle-aptitude students are permitted to take it and skip 
ahead if they reach criterion. Experience with the course has diown that the probability of passing the 
pretest is much greater for these trainees than for thelow-apttiudc trainees* The high-aptitude trainees with 
superior learning ability are also permitted to take instruction in large ncrements than middle- or 
low-aptitude trainees. In all cases, remediation follows incorrect responses to the program, whereas correct 
responses are always confirmed, or reinforced before the trainee continues through the course. The 
decisional model, therefore, has many fewer individualizing options than the Stanford model, but it does * 
capitalize on rate of progress through the course. 

Other CAI Applications, A number of other CAI installations, mostly university-based, have also 
implemented a variety of adaptive applications. At Florida State University, for example (Hansen, Dick, & 
Lippert, 1969), a CAI physics course was developed which incorporated individu^ reading, computer 
quizzes, audio lectures, single-concept film loops, and films. The flow from one medium to the next, as well 
as general progress within the course, was governed by the student*s achievement on the c<»nputer quizzes. 
Results of a comparison with the traditional lecture method indicated, in the first field study, that the CAI 
students were sfignificantly better on the final grade assignment than the students In the conventional 
method. A time savings of 17 percent was also effected and, while the conventional course achievement was 
marked by a gradual decrease in performance as the physics topics increased in complexity, achievement in 
the CAI conceptual problem exercises was markedly constant. 

At the University of Texas at Austin, a number of short courses have been developed. These courses 
range from the hard sciences (chemistry, mathematics) to the humanities (music, Arabic). A similar 
situation exists in public education utflizing CAI. The CAI Laboratory of the Kansas City Public School 
District, for example, provides adaptive instruction In areas such as scientific notation, mathematics, 
biology, and pollution. 

Modd Stnicture 

The ACA instructional system incorporates standard individualization components of learning rate, 
self-pacing, providing on line and off-line assistance, flexible utilization by the trainee, remedial cs^abilities, 
review frames, enrichment material, and behavioral modification variables such as Incentives, praise, and 
motivation. The two basic functional classes of the ACA models, pretask variables (set limits on the 
instructional alternatives) and wilhin-task variables (modifiable alternatives), can be designed into two 
adaptive concept sequenccs-a general adaptive model and a specific adaptive model. 
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Concept Adaptive Sequences. As discussed in the literature review section of this report, the 
within-task adaptation provides an accurate assessment of the student^s performance in a given program of 
Instruction. An optimal instructional task is presented to the student, and upon completion of the initial 
segment, the student Is tested. The test performance is evaluated in relation to the three type5 of 
classification error (overgeneralizatlon, undergeneralization, and misconception). If he has no errors, that is, 
he teaches criterion, then he continues the unit*s Instructional sequence; However Jf he does not make 
criterion, his responses, both right and wrong, are analyzed to determine the type of error being committed, 

General Adaptive Model. The first adaptive concept sequence^ termed general adaptive, prescribes a 
predesigned instructional program which follows the results of the initial test to determine if the student is 
committing a classification error. This mode! regulates the studeot^s instructional sequence as he progresses 
toward the terminal objective of a fiven unit of instruction* After the initial evaluation, each student's 
sequence of instruction is modified according to individual response patterns. For example* students who 
overgeneralize on the bc^nntng segment of the task would be presented higher probability instances with 
increased prompting. The number of Intermediate evaluations is determined by the concept difficulty* 
Some concepts may use only one sequence of examples fdlpwed by an exam, which would provide 
remedial help for those with errors. Another unit might Involve teaching several complex concepts, 
requiring several intermediate tests and remedial frames. 

Specific Adaptive Model. The second functional class of the ACA models is utilized in the specific 
adaptive model. The student would receive at the beginning of the instructional unit a presentaticm 
presumed to be optimal, followed by a intermediate evaluation. The within-task variables would be adjusted 
according to degree and type of classification error the student is making at this point. Degree refers to the 
measured severity of the error, that is, learners differ in the magnitude of incorrect re^onses. Whereas in 
the general model, the student would be ^ven a predesigned task to correct the error, the specific model 
would select a unique series of examples, in terms of difficulty and number of examples to correct the 
error. Thus, if a student was only making a slight ovcrgeneralization, his remedial instruction would use 
only a few examples, while a student making a gross ovcrgeneralization would receive a large number of 
examples. In each case, the decision parameters would ajust to the type and degree of error. 

Program Sequence Selection, In the various courses taught in the AIS, concepts vary in terms of 
complexity. In cases where concepts are difflcuU, it 's desirable to design units with multiple entry points. 
In such situations, pretask measures could be used to start the instructional presentation at a level of 
difficulty which is appropriate to an ir)dividual airman. For complex concepts omitting a pretask measure 
to flag appropriate entry points into the program, optimization would be limited to the use of remedial 
frames to correct errors. The pretask measure allows tow-aptitude or highly anxious students to enter a 
given program at a point which provides more instructional examples than a high*aptitude student. Thus, 
both pre* and within-task adaptations are necessary in complex concepts. On the other hand, the sole use of 
the pretask measure, would offer only a gross adaptation to the student's characteristics. While the pretask 
procedure adapts the presentation to the student^s entering trait capabilities, the within-task procedure 
makes the presentation self-modifying since it is continuously being adapted to the student's current 
response pattern and state levels. 

Interaction of Task and Student Characteristics, The pretask adaptive decision process which operates 
to enter a student into the unit of instruction for a complex concept at his optima! level is based mainly on 
an accurate evaluation of whether or not the student has in his repertoire the prerequisites to the unit. 
Although preskUl evaluation remains the most important component of the decision process, other variables 
also play a part in optimizing entry to the unit. Among these variables are aptitude indices (^.^.,'AQE), 
personological characteristics (e^., anxiety and curiosity), and cognitive styles. 

Once the student has entered the unit at his optimal level, these characteristics wiU interact on a 
frame-by-frame level with task variables to produce a given net amount of learning; or, at intermediate 
levels, to produce a set state of progress. In order to optimize this progress, therefore, the instruction must 
adapt to this Interaction between task and student characteristics. This interaction can be continuously 
monitored by the computer through an appropriate analysis of the student's cumulative response record. 
The basis then for the ^ecific adaptive decision process lies in a correct classification of the student's 
successes and difficulties as they are evidenced over time within the unit. Only if the decision rules 
effectively deal with this aspect of the process will prescriptive measures (including both remediation and 



enrichment be appropriate to an optimal progression through the unU. These decision rules may involve 
multiple factois, such as degree of correctness of the response, response latency, and cumulative indices of 
these two and other possible factors. The appropriate mix of factors which enter the decision rules will be, 
most probably, very heavily task-bound. That is, the optima! combinations will be different from task to 
task and will depend directly on the given task's specific characteristics. This optimization, or course, will 
evolve only through sustained formative evaluation of the decisional rules included in the model. In the 
meantime, however, a general, although less effective, approach can be taken. That is, broad decisional 
parameters can be established on the basis of the limited research evidence in instruction and theory. 

Model Variables , 

A paradigm of instruction for concept acquisition was discussed by Gagne (1970) and empirically 
Investigated by Tennyson, Woolley, and Merrill (1972). Concept acquisition was defined as the ability of 
the learner to correctly identify previously unencountered objects or events (or representations of such 
objects or events) as members or nonmembers of a particular concept class. It was assumed by Gagne 
(1970) and Tennyson (1972a) that for a given learning behavior^ an optimal information processing strategy 
can be identified. By manipulating task variables such as stimulus similarity, prompting procedures, 
sequence, and difficulty , an optimal instructional strategy for concept-learning can be designed. 

Instruclional Model 

The payoff of educational research is the application of the findings in an applied instructional 
environment. The purpose of this report is to demonstrate the feasibility of applying research variables on 
concept acquisition into a generalized adaptive instructional model for teaching concepts in the AlS jystem. 
This report does not present the methodology for the decision/selection stages in designing the actual 
instruction task; other sources give in-depth descriptions of those procedures (Tennyson, 1972a; 1972b). 
The purpose here is the presentation of the management model rather than the developmental procedures. 
The instructional model (Figure 6) is designed to accord with conclusions from research studies 
investigating those variables hypothesized to have a direct application to concept teaching. The 
instructional model's components are discusised in the following pajagraphs: 

(1) Pretest, The first component of the instructional model is a pretest on the concept class to be 
taught which assesses the student's entering behavior. The criterion referenced testing evaluates 
minimum capabilities. If the student meets criterion, he advances to Step (5), classification tc^sti 
if not, he proceeds with Step (2), definition. -'^^■''^^j' 

(2) Definition, In a study by Merrill and Tenn) son(1972) on prompting effects, it was found that 
subjects performed significantly better on the learning task when given the definition which 
identified the relevant attributes of the concept class (Markle & Tiemann, 1972) without the 
definition. The definition is a statement identifying the relevant attributes shared by a set of 
instances in a given class. Relevant attributes are enabling or prerequisite concepts assumed to 
be known by the student. Writing the definition requires a thorough analysis of the concept^ 
usually resulting in slmpUfication and reconceptualization of the class. 

(3) Review, Merrill and Tennyson (1972) included a treatment condition which presented the 
prerequisite subskills of the concept being taught. The results did not indicate that this variable 
was a significant factor In task performance. However* certain blocking schemes of the data 
showed that subjects with low pretest scores receiving a review did better on the posttest than 
similar subjects not receiving the review. The review component is, therefore. Included as a 
student option. In computer-controlled courses, students with low-aptitude profiles could be 
advised to t3ce the review. Whatever the rnodt? control, (he students should make the basic 
decision (see Bunderson, 1 97) , for a review on learner control) of whether or not to take the 
review. ^ 

(4) Instructional task. Tennyson, Woolley, and Merrill (1972) developed an optimal group 
instructional strategy for teaching concepts based on the theoretical work of Mechner (196!^ 
and Markle and Tiemann (1969, 1970). For concept acquisition, an optimal Information- 
processing strategy consisted of presenting examples and nonexamples to the student In such a 
way that the relevant attributes were clearly contrasted with irrelevant attributes. Task variables 
affecting learner processing of this information can be determined by four categories of 
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procedures. These categories arc (1) stimulus similarity variables, (2) prompting/feedback 
vjLTiables, (3) sequence variables, and (4) instance difficulty. More detailed explanations of these 
items are as follows: 

• Stimulus similarity variables include the following: 

• Matching of examples with nonexamples. An example is matched to a nonexample 
when both share identical or very simSar irrelevant attributes. 

• Divergent examples. An example is divergent Uom another example when the 
corresponding irrelevant attributes are different. Examples which share the same 
irrelevant attributes are said to be convergent. 

• Prompting variables include the foUov^ng: 

• Presenting a definition which identifies the relevant attributes (Step 2 of the 
model). 

• Using various devices to identify the relevant attributes embedded in exarriples 
presented in the task. 

• Explaining why a nonexample Is not an example. 

• S^<7U^rtcevan'aWe5 include the following: 

• Simultaneous presentation of instances. 

• Instructor*selected sequence. 

• Difficulty of instances. 

These four task variables are manipulated into an example set (Figure 7). According to the concept 
paradigm, two examples should be paired (divergent) so that they differ as much as possible in their 
irrelevant attributes. Within the same simultaneous presentation, two nonexamples are presented 
which are matched to their respective examples by having irrelevant attributes as similar as possible. 
This relationship of examples and nonexamples is designed to focus the student's attention on the 
relevant attributes. In the investigation by Tennyson (1972b) on the effect of nonexamples in 
acquisition, it was shown that subjects not receiving nonexamples responded randomly on the 
posttest, while subjects receiving nonexamples responded as hypothesized. 

Prompting is used in the example sets to explain why an instance is an example or why it is not 
an example. The subject matter determines the type and amount of prompting necessary. Example 
sets range in difficulty from easy (o hard. Depending on the adaptability of the program and the 
hardware, the instructional sequence could have multiple entry points and student control over exit* 
-Entry could be delermmed by student profile data to individualize on trait and state variables. 

(5) Classification test. Tennyson et al., ( 1972) designed a posttest which was capable of deteminlng 
the degree and type of classification error the student was making at the conclusion of the 
instructional task. The test examined the subject's scoring patterns in four different ways to see 
if he made an overgencralization, undergcneralization, or a misconception of the concept class 
(Markle & Tiemann, 1970). Construction of the classification test follows the same procedures 
as outline for the instructional task. The task presentation is expository, that is, the student is 
told if an instance is positive or negative. The classification test, however, is inquisith^e, that is, 
the student is not told the nature of the instances. Although feedback is given on the 
correctness of the answer, no prompting is given a wrong answer. Students meeting criterion on 
this test are finished with the lesson, while students failing to pass the classification test proceed 
to the next component, where they receive remedial instruction based upon the type of 
classification error they made on the test. 

(6) Adaptive sequence. Simple concepts would require only specific review if a subject fails the 
classification test. For complex concepts, it is possible to identify the type of ^u rfent error if 
criterion is not met (Tennyson et a!., 1972; Tennyson. 1972a). The v^o h.A: levels of 
adaptation that are.possible are(l) general and (2) specific. In the general ad r- r sequence, 
students arc classified into one of the three error categories. For each cMc ir. an optimal 
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group instructional task is pven to correct the crror. For example, if a student overgencralizcs, 
a specific program designed to correct that classification error would be given. The corrective 
programs would be as follows: 

• Overgeneralization. For students who overgeneralize, the general adaptive procedure 
would be to select instances of easier difficulty than normally would be used in a 
standard example set sequence used in the instructional task. Also, an increased level of 
prompting is given for each instance. 

• Undergeneralization, This §rtor indicates that the student failed to identify difficult 
examples. To correct this, the example sets would begin with harder instances than used 
in the instructional task. The sequence would basically concentrate on difficult example 
sets. 

• Misconceplhn, Since the subject seems to be focusing on some irrelevant attribute, the 
divergency of the examples would be expanded so that common irrelevant attributes are 
practically eliminated. 

In all three corrective programs, the students with each error categoiy would receive the same 
modified sequence. 

Specific adaptation is similar to the general adaptive condition in that adaptation Is made 
according to type of error, but the corrective procedures also are individualized according to 
the degree of error. The degree of error is determined by the number of err5^of a given type. 
A student who makes numerous overgeneralization errors would be given easl^instances than a 
student who only makes a few, The specific adaptive sequence also would increase prompting in 
a controlled situation so that no student is either overloaded or insufficiently instructed. * 

(7) Adaptive test. This test is designed to evaluate the effect of the corrective sequence. Test items 
would reflect the type of error to be corrected. It would not be a comprehensive test unless 
that degree of error was committed. Passing this (est would exit the student from the program. 
Failing again, the student would receive one further level of remedial instruction. 

(8) Specific review. This form of correction has a long histo/y in the field of programmed 
instniction. Remediation Is specific to the item missed. Again the problem's degree of difficulty 
determines the amount of corrective review. Concluding this component of Instruction^afmd 
test is given. 

(9) Review test, A standardized test simflar to the classification test is given. A student failing at 
this point indicates that he has leaned almost nothing from the instructionfil task. In such a 
case, this review test again assesses his behavior to perform at criterion. If the student meets 
criterion, he exits; if not, a continuation in the course is decided, 

(10) Advisement, In complex courses, it is possible that some students would have difficulty v^th 
certain concept lessons. In such situations, two decision can be made-the student drops the 
course or he continues with the next lesson and reschedules this lesson for a later date. The 
student's Individual cumulative profile is a major factor in the decision process (Bunderson. 
197J), 



Selecting New Instances 

In any adapt^e situation, a large number of Instructional instances need to be available for immediate 
usage. Also, revisions of the content once the system is in operation need consideration. The instructional 
program described allows for the storage of a large supply of examples and nonexamples from which to 
select corrective sequences. A subroutine program developed at the Univerity of Texas (Bunderson, 1971) 
generates new instances for a polyomial task, including a probability rating. King (1971) of Florida State 
University developed a program that rates difficulty of prose materials based on scales of readability. These 
t.wo programs illustrate that introducing new material into the instructional system can be a compcment of 
the main program. 
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The instructional model for adaptive concept acquisition was designed according to theoretical 
assumptions supported by empirical research data. The model allows for nexibflUy and modification by the 
contractor developing courseware for application in the AIS educational system. Although the model 
specifics concept teaching, other types of behavior could use the same sequence, and probably a typical 
{nstrucllonal lesson would include various types of behavior. In such situations, research-based variables are 
available to adjust the model. In instructional projects where various behaviors are used, this model luight 
be a subunlt or a larger mana^ment model. The premises here are that instructional design sliould be 
decided by theory as much as possible, and that design comp^ments should represent a parsimonious 
approach to development. 

Recommendations 

The following recomm^ 

• First, it is recommended that the prc^osed ACA Model be simulated and ultimately field tested 
since It focuses on a primary requirement of training, namely, concept acquisition. In addition, 
the ACA Model includes the training processes of diagnostic testing, dermltional learning, 
adaptive sequences of examples, and reviews, 

• Further, the topics of adaptive prompting, feedback, and task performance-related concepts 
should be Incorporated within the ACA Model or become correlated models. Each of these 
variables should be systematically pursued within the AIS research phase. 

• The use of pretask measures (trait measures) should be extended and utilized so as to increase 
the training efficiencies. 

0 • The most cost-effective use of media and instructional resources should become an operational 
component within the ACA Model apptoach. 

• The ACA Model should be extensively employed within* each of the three AIS courses, and 
appropriate evaluative comparisons should be made. 



V. COMPLEX TUTORIAL MODEL: RULtt -LEARNING AND PROBLEM-SOLVING 

Characteristics of a Complex Learning Model 

Much of the instructionpresentedin ATC courses concerns the acquisition of what Gagi)c( 1970) has 
called intellectual skills. Probably the most comnion type of Uatnlng undertaken by an aim.: n is the 
acquisition of principles or rules. After he learns tJie application of principles for a rule, hi'^ sul^sequent 
behavior may be governed by that mle. The use of such rules allows an individual to rc5v>ond to an 
enormous variety of situations in a consistent and effective manner. 

M.D. Merrill (1971) defines a rule as a ''statement of relationship between two or ino . concept 
classes," A student must show the relation^ip between these classes in order to demonsfrai^- ih^i he has 
learned the rule. It is not sufficient for the student to merely memorize the verbal sfateaicnt o( the rule. 
According to Gagne (J970), a rule is learned if a student is able to respond to a class of snwuhs situations 
with a class of performances, the latter being predictably related to the former by a class ot febf! >ns. M. D. 
Merrill (1971) further stales that the shident sh«ild be able to show (he rehtionsln^ bUM^eca the 
component concepts or classes in an unencounttred situation in Nvhich the given ;jIc is relevant. 
Rule-govemed behavior or nile-application behavior can be thought of, in computer sctcvcc km$, as the 
abOity to perform a spcciried operation on Incoinir.g data f/om a specified class of ioputs to produce a 
^ecilic output from a class of outputs. The Inputs may be thought of as the domain of the rule which 
consists of elements from the concept classes which compose the rule. The outputs njay be considered as 
the range of the mle which Is bounded by the operarion and perniissible Inputs. 
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M.D. Merrill (1971) describes problem-solving as lhat behavior which occurs when the student is able 
to select rclevaiU rules for an effective solution strategy when preser^ted with an unencountered problem 
situation for which the rules are not specified. The problem situation is one in which the analysis of several 
rules 'dn.t synthesis'of a strategy for solution is required, Gagne(1970) states that problem-solving can 
be VH^v ' jn i process by which the stud<?nl discovers a combination of previously learned rules that can be 
applied f > obtain a solution for a novel problem situation. 

1 5 V tghcr the type of learning required to achieve a stated objective, the more complex the 
instruct]* n becomes. To further complicate matters, an increasing number of interactioi^ variables such as 
featufCi • ; the subject niatter, characteristics of the instructional process, and ihe student's characteristics 
combtne to require quite sophisticated instructional models. In this chapter, the major emphasis will be on 
an adaot''^^ jnodel for teaching rules. It is anticipated that this model may be extended and adapted for 
ypplica(K*n in the teaching of problem-solving skills, f lowever, such an extension is not described he/e. The 
lattct pAit of this chapter does contain several implementations of highly interactive instructional programs 
tor tea^ ii problem-solving behaviors In a training environment. These instructional programs for teaching 
problem <^olving are not elaborated in great detail, but they are highly suggestive In iems of AIS 
applicano ^ 

Tiic adaptive instructional model for rule leatning under consideration in this chapter will utilize a 
combina'a/) of multiple-regression techniques and heuristic decision rules to select and sequence 
instnictinoal components into an idiosyncratic instructional strategy for a given student. The Instructional 
components will be maintained and stored in interrelated and cross-referenced component pools. The 
instruc al components will be selected from the appropriate pool and sequenced into an initial 
instructional strategy by the regression mode! and decision rules. Task characteristics such as rule difficulty 
and exarnplc ditftculty, and the student's profde of personological measures such as cognitive abilities and 
learninK style will be utilized to predict and specify the initial instructional strategy. Additionally 
within-task performance measurements such as number of items correct and test Item response latency will 
be collected. These measures will then be utilized by the regression model decision rules to update the 
instructif^nil strategy as necessary. 

"i' c Lpecificallon of instructional strategies for subsequent rules will be based on the student's 
perfon t iice und^r previous instructional strategies, it is anticipated that, through the use of this iterative 
cybirnr^^ic-adative procedure, an optimal instructional strategy of rule-learning for a given student will be 
approxfi dfil over a short series of rules. The model also incorporates a provision for supplementary 
xcmcdm jo<(ruction for students who do not have the necessary prerequisite skills related to component 
concepts uii J lower order rules. 

♦ . 

I r.c following subsection describes the role of rules and problem-solving in MS. The theoretical and 
experimental literature which support the model under consideration is presented in the succeeding 
subsection, whOe subsequenct subsections describe the model in greater detail. The instructional 
compoivjais and consequent instmctional strategies are also described. 

The Rok of ( ofppfex Tutorial Models in AIS 

A r .laMvcly major portion of each AIS course involves instruction in both rules and problem-solving. 
While nvA'.h of the fundamental information of a course is conceptual or p^chomolor in nature, the use of 
this in!\>rmntion requires the application of rules and the solving of problems. 

Tin election and completion of appropriate forms, item accounting, computer usage, stock control, 
and equipment management in the Inventory Management Specialist course all require the learning of rules 
in orderto complete defined (asks successfully. Further, as difficulties are encountered while completing 
tasks, students are required to apply sets of rules to solve the problem and to complete the job correctly* 

The vast majority of the learning taking place in the Weapons Mechanic course is at the rule level. 
Once pftf{<i of weapons have been identified, defined sequences of activities are presented as rules for 
disassembling and assembling each piece of equipment. Rules are also used in the teaching of systems 
functioning and safety practices. Problem^solving is an essential feature of the troubleshooting activities 
which occur both in the electricity blocks, lioklcss feed system blocks, and weapons systems blocks on the 
major aircraft. * 
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The Precision Measuring Equipment course is designed to develop the student's knowledge and 
understanding of the principles (rules) of measurement with an emphasis on precision and accuracy in their 
use. The students are taught to calibrate, trouble^oot, and repair approximately 200 items of precision 
measuring equipment. The troubleshooting phase of instruction is basically training in problem^sdvlng. 

It is, therefore, essential that the instructional paradigms developed for the AIM specifically consider 
the kinds of instruction that are best suited to the training of students In the learning and application of 
rules, and the solving of problems» 

Literature Review 

The review which follows describes the literature related to rute-learnhig and problem-solving and is 
divided into segments describing instructional strategies, task variables, and student characteristics. 

Instructional Strategies or Pawdfemj. Gagne (1970) has described the conditions which he feels are 
required for learning rules. The first condition or prerequisite for learning a rule is a knowledge of the 
concepts which make up the rule. Knowledge of these concepts Implies that the student must be able to 
identify or classify appropriately examples of the component concepts. The conditions required for 
rule-learning within the learning situation Include! 

t * A statement of the general nature of the performance to be expected when the learning Is 
completed 

• A presentation of verbal instructions which would Invoke recall of the component concepts 

• A presentation of a verbal statement of the rule 

• A presentation of a situation which requires the student to demonstrate or apply the rule. 

• Reinforce correct application of the rule. 

These conditions imply an instructional strategy or paradigm for teaching rule-governed behavior. 

Evans, Homme, and Glaser (1962) have developed a Ruleg system for the construction of 
programmed learning sequences based on niles(ru*s) and examples (eg^s). Rules are defined as statements of 
some generality from \^ich substitution instances or examples can be obtained. This may Include su<ii 
things as a definition, a mathematical formula, an empirical law, a hypothesis, or an axiom. An example is 
defined as a statement of some specificity which is derived from tfie more generalized rules. The Rulcg 
system also Is made up of additional components which are defined below with their corresponding 
shorthand notation: p 

ru An incomplete or partial rule vAtkh requires the student to respond by completing the rule 

eg An Incomplete or partial example which requires the student to respond by completing the 
examples 

ru- A terminal situation where criterion behavior is required with minimum stimulus support, and 
^ the response is usually a statement of the rule 

eg A terminal situation where criterion behavior is required with minimum stimulus support, and 
the response is usually the solution of an example problem with no prompts from eiUier rules 
or examples 

eg A false or nonexamplc of the rule. 

Evans at at,, (1962) describes several different frame types which are made up of these components. 
The following instructional strategy or sequence of ruleg frame is reconmendcd under this system: 

• Be^ with a ru + eg + eg frame which presents a verbal statement of the rule, a complete 
example of the rule, and a partial example which requires a response from the student. 

• Gradually withdraw stimulus support with the use of such frame types as ru + eg, eg + eg, and 
eg+W, 

• The instrupijpnal sequence should terminate when the student can deal effectively with ?g 
frames andW frames. 
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• When two or more rules arc being prescnted,Tu| + ru^ and eg| + c^j frames should be used to 
help the student discriminate between the two rules. ^ 

It is further suggested that, to insure adequate generalization, the examples corresponding to a given 
rule should be as diverse as possible, with the first example being the simplest possible nontrlvial example. 

Task Variables. There are many task variables which affect the efficiency and effectiveness of an 
instructional strategy or sequence for teaching rule-learning or problem-solving. Those variables considered 
to be of major importance are described below. 

• AyailabilUy of Verbal Statements of the Rule, Both Gagne (1970) and Evans et al,, (1962) 
recommend that the student be presented verbal statements of the rule at the beginning of the 
instructional sequence. The presentation of the rule at the beginning reduces the risk of having 
the student induce an Incorrect rule. The verbal statement also serves as a cue to the learning of 
the new rule. A study by P. F, Merrill (1970) revealed that the presentation of verbal 
statements of the rules reduce the number of examples and the amount of time required to 
learn rules,. The presentation also increases performance on a transfer test. 

• Number of Examples. The specification of the number of examples to be used is dependent 
upon the difficulty of the rule, the abilities of each individual airman, and the amount of 
generalization required. It is antfcipated that this task variable will be one of the major 
individualization parameters. The optimal number of examples required in a given sequence can 
only be estimated prior to the first administration of the program. 

• Type of Examples. The examples used with each rule should vary in difficulty, and be as diverse 
as possible from one another. These variables have been ^own to be of considerable 
importance in research with concept-learning (Tennyson, Woolley, & M. D. Merrill, 1972), but 
little empirical research has been done with rule learning. M. D. Menill and Tennyson (1971) 
also have found that the availabilit^f negative or false examples is an important variable in 
concept learning. The importanc^Bnegative Instances in the learning of rules has not been 
established at the point. ^ 

• A nyount of Practice, This vaiable is concerned with the number of times a student should be 
required to apply a given rule. It is anticipated that this variable is related to individual 
differences, the complexity of the rule, and the uniqueness of various cases of the rule. 

• Memorization of the Verbal Rule. Gagne (1970) emphasizes that memorization of the verbal 
statement should not be considered as rule-learning. However, it is possible that the ability to 
state the rule verbally would allow the student to be able to talk about the rule on a later 
occasion and may serve as a valuable cue and/or memory aid for subsequent application of the 
rule. 

• Prompting. Some students have difficulty realizing the relationship between a verbal statement 
of a nile and its application in an example or problem situation. M. D. Merrill and Tennyson 
(1972) found that the presentation of written prompts which point out the relationship 
between concept instances and the corresponding definition of the concept facilitate 
subsequent classification behavior. Although this variable has not been studied in rule-leaming 
situations, it is anticipated that the availability of prompts will be an important variable in 
rule-learning. 

• Amount and Placement of Review. The results of two experimental studies conducted by Gay 
(J 971) showed that one review is more effective than no review, regardless of the temporal 
position in the retention of mathematical rules. However, optimal retention over a 3-week 
intenral is obtained with two reviews, one early and one delayed. 

• Student Characteristics. P. F. Merrill (1970) investigated the interaction of cognitive ability 
with the availabiliiy of rules in learning an imaginaiy science by computer-assisted Instiuction. 
He found significant abflity by treatment interactions uring test item response latency as 
criterion and individual reasoning tests as co-variables. Reasoning had a high negative 
relationship to test item response latencies for Ss in an examplq-only group, but this 
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relationship was significantly smaller for S% who received verbal statements of ih:^ lules in 
addition to examples. Therefore, Merrill concluded that the presentation of ru?cs ^^ffocted a 
reduction in the requirement for reasoning abflity. Dunham and Bunderso i (1969) 
investigated the effect of decision-ruk instruction upon the relationship of cognitivr- .'iDiJities to 
•performance In concept problems. Tlie results Indicated that different abilities were important 
In the two conditions. Associative memory and induction were important ir. th^ no rule 
instruction condition, whUe general reasoning and induction were important In the 
decision-rule Instruction condition, 

Gagne (1970) argues that one of the main individual characteristics affecting rule-Icariitng it the 
.Individual's ability to perform the prerequisite Intellectual skills which are subordinate to the new rule in 
question. Therefore, an efficient instructional strategy must be able to adapt in such a way as to provide 
instruction for students who do not know the component concepts of a rule or are not able to apply rules 
which may be subordinate to the new rule to be learned. 

Gay (1969) investigated the effectiveness of using a prclnstruction retention index to predict the 
number of examples each student should receive in learning mathematical rules. The retention index was 
based on an empirical determination of the optimal number of examples each individual needed in order to 
maximize his retention of preinstructlonal rules. After determination of the retention index, all subjects 
were randomly assigrted to one of three treatments - (1) the subjects in the variable example group were 
prcserttcd mathematical rules in which the number of examples presented for each rule was based on the 
subject*$ optimal number of examples on the prclnstruction task, (2) each subject in the choice group was 
allowed to determine the number of examples he would receive for each rule, and (3) the subjects in the 
fixed group were given exactly three examples per rule. The results revealed a significant sex-by4reatment 
interaction. Females in the variable example group performed significantly better on both immediate and 
delayed retention measures than females In either the choice of fixed group; However, males in the choice 
group performed better on both retention measures than males In the other two groups. 

Recent research into the teaching-learning process has focused on (he learner as an active analyzer 
and synthesizer. DiVesta and his associates at Penn State (1970, 1 97 1) in formulating an "Evolving Theory 
of Instniction," have stated that: *'No matter what message is presented or what teaching method is used, 
instruction provides only potentially effective stimuli for the student. If the message Is to become effective, 
the learner must be motivated to attend, and he must have a cognitive structure requisite for appropriately 
analyzing the message." (DiVesta, 1971 , pg. 1) 

The learner, therefore, brings to the learning process styles and strategies which enable him to learn. 
Studies (DiVesta, 1970) Investigating two states of the theory which relate to learner style (analysis of the 
instructional messages and synthesis of the learning outcome) have provided the following information 
about learner strategies'. 

• Evidence was presented that the learner's prefeience for visual or auditor^' modality determines 
which features of the instructional message actually are recorded in the learner's sensory 
register. 

• Motivations exist to seek out particular parts of a message for further attention. 

• The motivational effects of stimulus characteristics vary in a predictable manner contingent 
upon the learner's immediate prior experiences. 

• Provided the learner is motivated and the instructional message registers, then particular 
inslmctional methods seem to facilitate the learner's analysis acttvuies. Instances of facQitative 
instiuctional methods include: 

• Providing adequate contextual cues within the message 

• Instructing ccmimon elements of the message together. 

• Authoritative guidance has differential effects on analysis in problem-solving situations, 
contingent on the dogmatism of the learner. 

Several additional studies are in progress dealing with the second, or synthesis, sta'ge. These relate to 
the effects of different testing procedures on the learner's interpretation of the desired instructional 
outconie; differential effects of anxiety and tasks demands on desired outcome of creative responses; 
♦Mntformational processes; and instrumental activitk5s(c^., note-taking behaviors, recitation ^exercises). 
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Descriptk)n of the Adaptive Model for Rule-Learning 

The adaptive instmctlonal model for rule-learning proposed in this section utilizes linear regression 
models and heuristic decision rules in an iterative cybernetic process to select and sequence instructional 
components in instructional strategies which are idiosyncratic to individual students. The instructional 
components will be selected from interrelated and cross-referenced component pools. The components to 
be utilised in this model are an adaptation and extension of the Ruleg system. (Evans 1962): 

ob-Dlsplay of a verb^ statement of the behavioral objective related to the rule 

ru-Display of a verbal statement of the rule 

ru-An incomplete or partial statement of the rule v4iich requires the student to respond by 
completing the nile 

nf-A terminal situation which requires the student to state the rule with minimum stimulus support 
eg-An example of^the rule 

'c^-An incomplete example or problem which requires the student to respond by completing the 
example or solving the problem 

terminal situation in which the student is required to solve example problems with minimal 
stimulus support 

eg-A false or negative example of the rule 

pr-Prompt, a verbal description of how the example relates to the rule 

fb-Display of feedback concerning the correctness of a student*s response to a problem or partial 
example " 

n(eg)-A scries of n examples of the same rule 

n(eg + pr)-A series of n examples with a corresponding prompt for each example 

ru 1 -A verbal statement of Rule 1 as contrasted with Rule 2, etc. 

eg2-An example of Rule 2 as contrasted with an example of Rule 1 or Rule 3 

n(eg + fb + pr)~A series of n example problems with corrective feedback for each problem and 
prompting for ^ose problems answered incorrectly. 

According to the proposed model, these instructional components would be selected from* 
component pools and integrated into a sequence of instructional frames. A sequence of such frames for 
teaching a g^en rule may be thougjit of as an instructional strategy for that rule. It is obvious that the 
instructional components could be combined in an extremely large number of different frames which 
could, in turn, be sequenced in many different orders. However, on the basis of the theoretical and 
empirical studies reviewed in this section, an instructional sequence for rule-leaming is proposed In Figure 
8. Each line in the figure represents a frame composed of instructional components described previously. 
The instructional sequence consists of the successive display of frames l>eginnlngat the top and proceeedlng 
to the bottom of the figMre. This strategy presents an objective, rule, example, example problems, 
corresponding feedback, and prompts. The supporting stimuli such as objectives, rules, examples, and 
prompts are faded gradually untfl She student is able to s(^ve problems and state the rule with minimal 
support. The model assumes that only very low-ability students would need to proceed through every frame 
in this exhaustive instructional sequence. Most students would be able to skip some of the fading frames, 
whfle very high-ability students might be able to skip the highly prompted frames and some of the fading 
frames. The instructional sequence or strategy for a given student will be determined through the use of 
linear regression models and ht^nristic decision rules. 

A proposed review and Integrative sequence or strategy is presented in Figure 9. This sequence would 
be presented after several rules have been learned. Again, the extent and frequency of this review strategy 
would be determined on an idiosyncratic basis by the adaptive model. 

Figure 10 is a flow diagram of the overall adaptive instructional model for rule-leaming. When a 
st^ent enters the rule-learnlng model, it is necessary to determine if he possesses the Intellectual skills 
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Fig. 8, Sample Instructional Strategy for Rule-Learning, 
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Fig. 9. Sample Instruction Strategy for Review and Integration. 
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which are prerequisite to learning the new rules. If the student is unable to apply prerequisite rules or 
classify instances of component concepts which make up the new rules,' then he should receive 
supplementary remedial Instruction. It is also necessaiy to obtain data concerning personological variables 
such as cognitive abilities and learning style which may be useful in predicting an appropriate initial 
instructional strategy for each student. These data, along with data concerning task characteristics such as 
the difTlculty level of a given rule and correspoi^ding examples, will be utilized by the linear regression 
models and decision rules to select instructional components to make up an initial instructional strategy for 
a given student. The generated instructional strategy is then presented to the student. Wi thin-task 
performance data, such as display latency, number of items correct, and test item response latency, will be 
collected as the student proceeds throu^ the generated instructional strategy. These data are then utilized 
by linear regression models and decision rules to modify or update the instructional strategy as necessaiy. 
For example, if a student is performing poorly under the instructional strategy, it will be necessaiy to 
present additional frames with a high level of prompting and a greater number of examples with a slow 
removal of stimulus support. On the other hand» if a student proceeds very rapidly through the displays and 
solves all problems correctly, it may be possible to remove stimulus support more rapidly and to present 
fewer examples and/or prcxmpts. The student's performance on the terminal items for a given rule will then 
be utilized to specify the instructional strategy for subsequent rules. The student's performance on previous 
rules is also utilized to specify the frequency and type of review and integration strategy which will be 
utiltzed for a given student (Figure 9). TTiis iterative and cybernetic procedure is followed until all rules for 
a given series are learned. 

The following have been identified as possible input variables to the model: 

• Difficulty level of a given rule 

• Difficulty level of examples 

• Number of rules in a series 

• Entering cognitive abilities such as general and inductive reasonings ^ 

• Prelnstruction retention index 

• Learning style, such as dogmatism and modality preference 

• Within-task performance measures such as display latency, number of items correct, and test 
item response latency 

• Within-task state variables such as state anxiety and subjective confidence. 

The adaptive instructional model for rule-learning will manipulate the following instructional 
variables as output: 

• Number of examples 

• Type of practice problems 

• Number of practice problems 

• Level of prompting 

• Rate of stimulus support fading 

• Amount and placement of review and integrative rnaterials. 

The adaptive instructional model for rule learning presented in this section is based on the theoretical 
and research literature available at this time. However, the model contains many innovative features which 
have yet to be implemented and validated. Considerable research will need to be done in order to refine the 
model However, it is felt that the model presented in this section is a rational starting point and wili 
provide the framework in which to make future refinements and to investigate those variables which are 
crucial in the instruction of rules. A simulation of this model would require further dellneatton of the linear 
regression models and heuristic decision rules required to predict and specify appropriate instructional 
strategies. The input variables and instructional decisions listed previously may be expanded or revised 
based on the properties and constraints of the heuristic decision rules to be developed. Further baste 
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research into the role and. effects" of number of examples, prompts, type of examples, amount and 
placement of review, and rule frame structure and sequence will facilitate the generation of appropriate 
regression models and decision rules. 

Description Models for Problem-Solving 

The cunrent state-of-the-art only allows a very tentative statement of an optimal instiuctional strategy 
for leaching problem-solving behavior. According toGagne(1970) the following conditions are required for 
prob!en>solving. The student must be able to recall the previoudy learned rules which are relevant to the 
problem solution. In the learning situation, he suggests that verbal instrucllons should be used to help the 
learner recall relevant rules in close time succession. He further suggests that verbal instruction should be 
used to guide or channel the thinking of the learner in certain directions. This guidance should always come 
short of describing the sdution itself. However, as a minimum, it should include a description of the goal 
and the general fonn of th solution u\ order to limit the range of hypotheses, 

Althou^ Gagne stales that repetition docs not appear to be an important condition for 
problem-solving, since what is learned is highly resistant to forgetting, it seems apparent that the 
educational process should provide increased opportunities for students to be involved in problem-solving 
situations^ By increasing students* exposures to a wide variety of probl/?m-solving situations, their skill In 
solving unique problems will be increased. 

Gagne (1966) also suggests that several individual differences may be related to problem-solving 
behavior. Applied to the AIS setting, these individual differences could be stated as follows: 

• The number and variety of rules relevant to problem solution that a student may call upon 

• The student's ability to recall relevant rules 

• Differences in a student's concept distinctiveness 

• Ability to generate hypotheses 

• Ability to match specific instances to a general class in oidtr to verify solutions. 

Perhaps the most fully developed model that could be applied to the teaching of rules and 
problem-solving is Task Teach (Rigney , 1969), a computer time-sharing system used to assist the learning of 
a variety of tasks, including operating equipment and troubleshooting problems in the equipment. A 
defmite distinction is made between two different types of serial action tasks. The first type include 
prescriptively guided tasks which can be completely specified before they are performed. Included in this 
type is the operation of specific test equipment. In contrast, a problem-solving or troubleshooting type of 
task would have choice points and decisions required for completion. Both linear tasks and intricately 
branching tasks (where the next subtask is contingent upon the outcomes of previous subtasks) may be 
handled in the Task Teach Program. 

The main adaptive fealuie of the Task Teach Program is the high degree of learner control. The 
learner is allowed to operate in any of four differenr modes which provide varying levels of support and 
guidance. Mode 1 provides maximum guidance. Mode 2 provides moderate guidance, Mode 3 is a self-test, 
whfle Mode 4 is a final examination of the stvdenVs performance. In addition, several commands are built 
Into the program which allows the student to have extensive control over the sequence and amount of 
support he receives. The "map" command gives the student a diagram of the task structure which identify 
the action goal hierarchy he must accomplish. For problem-solving or troubleshooting tasks, a decision tree 
may be represented. Under **monitor" support, the Task Teach Program automatically monitors student 
errors and gives the student appropriate feedback. The "progress'* command gives the student a list of 
remaining possible malfunctions or alt goals accomplished up to that point. The '^explanation" command 
gives the student reasons why certain tasks should be performed within certain subgoals. The *'next'* 
commar\d the next actions which should be performed or reviews how to perform them. The ''history*' 
command gives a response-by-response record of everything the student has done to that point. The 
''restart" command allows the student to start over or switch to a new task, while the '*quit'' command 
allows him to sign off the system. Within each of these commands the amount of prescription versus 
interactive materia) varies. 
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The present student interface is a teletype terminal with a student-operated carouse) slide projector. 
However, plans for interfacing the slide projector with the terminal, of using CRT devices, and also for 
Interfacling the actual test equipment or simulated training aids with the computer are being considered. 

The Task Teach Program has general applications an^ could be used with any serial-type task. Efforts 
have been made to simplify the use and entering of the parameter data by the Instructional designer 
without having to learn a CAI language. While all that is required is the input of lists representing the 
structure 6f the task, a sophisticated task analysis of the task must be conducted by the Instructional 
designer prior to entering the lists. A knowledge of the list format is also required. It is not necessary, 
however* for the author to program all possible task lists since the student control commands are used to 
call up separate subroutines. 

This approach has very definite application for Air Force courses, particularly PME which entails 
considerable equipment operation anSd troubleshooting activity. Such a program would provide considerable 
opportunity for the student to practice operation or troubleshooting skills with only necessary assistance 
and feedback. 

Simulated problem-solving experiences can pr6vide students with the opportunity to learn complex 
concepts as well as skills through participating in lifelike experiences without the high cost and rjsk involved 
in the actual task. In a computer-based instructional simulation, programming techniques are incorporated 
to monitor and analyze the student*s work in the performance of assigned tralning'tasks automatically. 
These techniques provide information for diagnosis of learning difficulties and lead to the development and 
testing of hypotheses related to overcoming the problems. 

Feuizeig and Lukas (1971) describe two computer-based simulations for complex operational tasks 
which incorporate this feature. The first task implemented and tested involved perceptual and motor skills 
to maintain an aircraft in a holding pattern solely from information provided by instrument indicators, A 
monitor system for this task was implemented and used with both new and experienced pilots. The general 
approach was to design a highly task -specific system of programs, called an Instructional monitor, for 
following a student's work and diagnosing his difficulties along the way. Such a monitor requires detailed 
information on the various kinds of errors possible. Procedures for diagnosing specific errors in terms of 
observable effects, and associated information about possible reasons for errors in faulty procedures or 
conceptualization. This information provides the basis for instructional decisions. In the simulation, the 
trainee was given the task of flying a vehicle over a prescribed course on the basis of instrument 
information. The vehicle is represented as a moving point on a scope which also displays the appropriate 
changing Instrument information: altimeter, automatic direction finder, magnetic compass, and rate of 
tetnm indicator. The location of the vehicle, the ground track foi the course, the instrument indicators, and 
the trainee's actual flight path can all be displayed. The trainee controls the vehicle through the use of a joy 
stick linked to the computer. The problem may be complicated by the introduction of winds, drift of the 
controls, and variations in altihjde. As a student proceeds in his flight, the state of the forced dials and the 
position of the plane at each clock interrupt are stored in the computer. When he has completed the task, 
the trainee may review his flight in its entirety by calling on the replay monitor. This program plays back 
the flight dbplaylng both the instmments and the holding pattern along with ground track. The replay 
monitor also attempts to detect the trainee's local errors and to make appropriate comments. The 
monitoring and simulation system was tested and evaluated in an instructional experiment. The subjects 
had flying experience spanning over a wide range of h6uTS before the experiment began. The monitor 
systems was used to train trainees to fly holding patterns at the computer with some degree of accuracy, An 
actual fli^t testing of the trainees was then carried out to see if this skai carried over to the real iisk. The 
trainees showed considerable success in transferring from the computer-trainlng situation to the actual 
flight situation. They demonstrated facility with the use oYthe quipment and goodcomprehension of the 
pruiclples involved in performing the holding task under moderate wind conditions. The authors suggested 
that appropriate instructional programs of the same kind can also be designed for use with many other 
complex perceptual motor tasks. 

A second task area reported invdved the acquisition of perceptual skills essential to ship maneuvering 
and cdlision avoidance. This task area involved a graduated sequence of course-estimation problems 
presented on a simulated PPI screen and permitted both the gradual acquisition of skills and lsolatic« of 
conspicuous learning difficulties. Related studies provided a framework for detecting deeper underlying 
difficulties which show up in more complex realistic situations. While these studies used a very small sample 
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complex tasks involving mechanical and perceptual, as well as intellectual, components. This kiitd of 
extension is particularly important iji making many areas of technical training both feasible and more 
cost-effective. There is great potenUal in courses such as Weapons Mechanic for simulation of 
troubleshooting problems; problems both greater in number and with more dangerous consequences than 
could ordinarily be presented to trainees. 

Computer simulations have also been used to provide the student with lifelike whole job experiences. 
Direct Support Unit Simulation Exercise (Powe, 1969) Is one such example. In a dynamic representation of 
under-ftre conditions, the student is required to perform tasks as supervisor of a Direct Support Unit 
responsbile for missile system repair. Decisions are required based on given situations involving personnel 
operations, security, supply, repair of malfunctions, backlog, length of workday, maintenance of organic 
equipment, liaison, and mflltary unit training. The student bas available administrative and technical 
publications as well as supply and maintenance documents with his goal being to make logical decisions 
under realistic conditions. 

This type of simulation would be especially appropriate in the Inventory Management course where 
students need experience in seeing the entire scope of their occupation, which has typically not been 
possible for them in traditional training conditions. 

Each of the instructional programs for teaching problem-solving provides opportunity to learn 
appropriate techniques to approach finding the solution to a problem. It is obvious that application of rules 
in problem-solving requires knowledge of a variety of rules, the choice of the appropriate rule, and the 
correct application of the appropriate rule. To develop these skills, varying amounts of practice arc required 
iri simulated and in actual situations. U is on the dimension of amount of practice, as well as the dimendons 
of varying the amount and degree of prc^ipting or guidance during the instructional sequence,* that 
adaptation to individual difference might be made. 

Recommendations I 
The following recommendations are made: 

• Althou^ the adaptive model for rule-learning proposed in this section is based on current 
theoretical and research literature, many features of the model are innovative in nature ^nd will 
require additional research and validation. It is, therefor^, recommended that research be 
conducted in this area. In addition, the linear regression models and heuristic decision rules 
utilized by the adaptive model will require further dehneation and empirfcal validation. 

• A simulation of the adaptive model will facilitate the specirscation of initial decision rules and 
regression algorithms. The input variables and instructional decisions should be expanded or 
revised according to the constraints and characteristics of the simulated adaptive model. It is 
recommended that two versions of the model be siniulated-a full and a reduced mpdel. The 
fuU version model would require a highly interactive mode of instruction such as CAI. However, 
a reduced-version model could be utilized in a semi-interactive mode such as CMI. Therefore, it 
is recommended that both modeb be simulated. The reduced version would not require an 
updating of the instructional strategy according to within^task perfomiance data. 

• The applicability and extension of the adaptive model for rule-learning into the area of 
problem-solving should be investigated. However, it is anticipated that instructional models 
such as Task Teach might prove to be very useful in problem-solving applications in the AIS. 

• Provision for research into the role an effects of the number of examples, prompts, types of 
examples, rule frame structure, and amount and placement of review should be made in the 
implementation of the adaptive model. This research would provide data for the revision and 
refinement of the model. 
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' VI. ALGORITHMIC REGRESSION MODELS 

Adaptive instructional models have been in use since the Rrst self-instruction program. Learners have 
characteristically adapted to the learning envircHiment by choosing the form of studying optimized for 
them on some variables/These adaptations to accepted instructional technique more often than not were 
based on incomplete information, often false assumptions about the learner, and contingencies not related 
to optimum learning requirements. Recent research on learner characteristics, instructional variables, and 
aptitude treatment-interactions (eg., Tobias, 1969; Melaragno, 1967; Cronbach & Snow, 1969; Bracht, 
1970) has indicated the potential for adapting instructional techniques along many dimensions in order to 
match unique learner profiles more closely. Whereas only a few years ago, fine-grained, cybemetically 
controlled adaptations would have been difficult from administrative and operational points of view, the 
use of high-speed digital computers in the educational world has made the assignment of learner-sensitive 
instructional techniques and materials increasingly viable. 

Characteristics of Algorithmic Regression Models 

A number of instructional systems based on response- and learner-sensitive models have been 
attempted; some of these were dependent on the availability of a computer. Programmed Instruction 
(Skinner. 1961; or Crowder, n.d.) provided learner-response characteristics to learning tasks vrtiich utilized 
printed media. Individualized instructional programs such as Project PLAN and IPI have been avaOabte for a 
number of years. (Flanagan, 1970; Llndvall & Bolvin, 1966). In these programs, students are assigned 
instructional modules according to a predetermined scheme based upon reaching criterion on previous 
materials or on pretest scores. The student is required to progress through a modified **tiack" system in 
which the opportunity exists to **jump'* from one track to another. 

Suppes et qL, (1963) utilized a computer to present instructional materials and to make branching 
decisions on the basis of student-response patterns to the materials being presented. Computer-Managed 
Instruction (CMI) can also provide for the assignment of either new materials as the student reaches 
criterion, or remedial materials when he doesn't {e.g., Gallagher, 1970; Lawler, 1971). Although the 
adaptation described so far is responsive to the learner and appears to approximate the behavior of a \ 
teacher, the methods provide, at best, for multiple-track systems utflizing essentially similar instructional 
materials within each task. It is the purpose of the algorithmic regression adaptive models described in the 
next paragraph, to provide for a more comprehensive individualized utilization of media, materials, and 
diverse instructional tecTuiiques. 

The adaptive instructional model under consideration in this section utilizes multiple-regression 
analysis to provide a prediction of the media, content, and instructional method necessary to optimize 
learning. Of the adaptive models described here, regression modeling is unique in that it utilizes a number of 
learner characteristics to assign an optiihal instructional experience to achieve quantifiable criterion goals. 
Thus, to the extent that the variables input to the prescription output variables, this model can be expected 
to predict successfully the media, content, and instructional method best suited to the characteristics of a 
particular student. 

Markov models which have been suggested by Atkinson (1967) were considered tp be inapplicable to 
technical training adaptation problems. The models typically require that the lest items or responses vyhigh 
are given in order to make state-changing (branching) decisions be highly and independently homogeneous. 
This state of affairs is typically not the case as behaviors tend to be heterogeneous and highly related. 
Furthermore, the transitions which characterize Markov models are of a minute nature which could only 
transfer to the minute-by-minute course of instruction. Thus, this framework is clearly too detailed and 
complex as an intertreatment decision process. 

The selection of a regression approach for this model is prompted by the quantifiable nature of the 
prediction algorithm. For the sample of students on whom the prediction model is developed, the error of 
prediction (represented by the "distance" between the student's actual score and the score predicted,p<?i^ 
hoc, for him by the model) is at its minimum due to the nature of regression analysis. Regression is 
typically utilized to investigate the interrelationships between predictor variables such as age, ability scores, 
or SES, and criterion variables such as success in college or course grades. In the present case> predictor 
variables such as ability^ aptitude, personological variables, and treatment characteristics are used to predict 
criterion score on a multitude of course performance levels in order to specify the pptimal instructional 
treatment. 
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The regression model being considered will use two classes of variables to select the optimal 
instructional altemativei These input variables may be classified as either trait oi state in nature. Trait 
variables may be characterized as static, long-lasting, and descriptive of the individual's behavior over 
longtime periods, ie, as indices of expected general behavior. State variables, in contrast, may be 
characterized as dynamic, short-terifi, and descriptive of the individual's behavioral nature m^n specific 
situations, at particular times, and over shorter time periods. 

The trait, historical, or a priori variables which may provide input to the regression models include 
both background indices such as age, SES, and prior knowledge of similar material, as well as more 
traditional measures such as IQ, trait anxiety, curiosity, and motivation, personality measures, aptitude 
indices, and achievement test scores. These trait variables may be considered '^historical" since they are 
available prior to instruction and are not likely to be updated during the weeks or months of ongoing 
instruction. 

The input of state variables to the models include primarily responses and latencies' on 
criterion referenced test items and traditional achievement test scores, which may be required to assess the 
student's current leaming/aptitude state. Additionally, measures such as state anxiety, curio^ty, and 
motivation may be used to "correct" for current changes from the learner's trait levels. 

An additional characteristic of regression models Is that multivariate techniques are applicable. These 
techniques permit the foint analysis of the relationships between the vector of input ^md the vector of 
output (prescriptive) variables as well as allowing for the prediction of the prescriptive variables under 
statistical assumptions less stringent than with univariate techniques, making the techniques generally 
applicable to data in the AIS setting. 

Issues to be considered in the use of regresrion models for adaptive instruction include (1) the 
determination of the statistical properties of the variables, (2) the selection of the instructional components 
to be predicted by the regression equation, and (3) methodological requirements for establishing sufficient 
(Simon, 1969) criterion levels where required. 

The deteimination of the statistical properties of the variables is an important consideration in that 
the distributions of scores can have a marked effect on the accuracy of the prediction. Markedly skewed 
score distributions, particulariy when accompanied by small variances, lead to a smaller correlation between 
the linear combination of scores and the criterion to produce lessened effectiveness of the model. The range 
of scores can have a similar effect on the predictive accuracy of the regression (or algorithmic) model. • 

The instructional parameters or components to be predicted by the regression equations are selected 
on the basis of three criteria. The first criterion is the availability of research evidence to support the 
relationships between the available trait and state indices and the prescriptive variables. Givcn the current 
state-of-the-art, if one or more research studies judged to be reliable exists, this criterion will be assumed 
met. The second criterion relates to the usefulness of the criterion classification to instruction, eg., media 
choice is useful, but typeface choice is not. A third criterion relates to the inclusion based on iterative 
empirical validation. The variables which are being ccvisidered are listed in the model structure subsection 
of this section. 

Prior to predjcring the c^timal level of an instructional prescription factor, it isnecessaiy to establish 
acceptable criterion levels for each instructional factor to reOect an efficient placement of students. The 
methodcJogy for detemiining the criterion levels will be selected during the simulation phase of the project. 

Applications in the Advanced Instructional System (AIS) 

The primary goal of AIS is to provide a prototype, state-of*the-art, Individualized, multimedia, 
computer-based instructional system which demonstrates cost-effective training procedures. The 
characteristic of multiple regression analysis for predicting optimal instructional parameters which most 
clearly relate to the cost-effectiveness constraint is the "least-squared error" solution. This solution of the 
regression equation provides for minimal differences between the actual scores on criterion measures and 
predictions of the scores. On the same criterion measures, the empirically derived regression equations 
provide for the closest match of predicted and actual scores and can be expected td prescribe the most 
efficient instructional treatment for the operational group of students. 
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The iolution of the regression equation will be determined througli an iterative procedure in which 
initial groups of studer^ts are landomly assigned to instructional treatments on a stratified basis. The 
characteristics of these students and the treijtment. given will then be analyzed and related to the 
performance levels obtained. Succeeding gtoups will then be assigned to tnstructiona! treatments leading to 
the best predicted score by means of the regression equations developed. Periodical updatings and revisions 
of the model will result in renncmenl of the roles of various predictors elimination, addition, or 
change of contribution of individual indices) and a maiiitenance of the nioders efficiency. 

RoleinAlS 

Ihc role of algorithmic regression models wiiyun AIS is gene/ale the adaptive instnjctional 
prescriptions. Wliereas other models described in this report may be characterized as operaUng at an 
intratask level, the regression models operate at an intertask level. As such, the model is the mechanism for 
predicting (and thereby selecting) the optimal media, difficulty, and/or iiiStructional method for each 
learner. It is expected that the complete range of predictive choices will rarely be available^ but that an 
array of ins^n/ctional ahernahves wji} be available at each choice point. Upon the choice of a nonoperablc 
prescription, the closest avalabic option would be >*ckctcd and offered to the student. 

Tim framework for computer-based prescription may be placed midway on a scale witli end points 
c!iar;)cteri7.ed as ''response sensiiive*' and "respond insensitive." A response-sensitive paradigni would 
provide for prescription of detailed ^'frames" of instruction following eacli student's response, whereas a 
iLSponse-insejisilive paradjg;n would provide for, at bcst,a track system in wiiich only gross adaptation 
v;6uld occur. The envisioned CMi fran^ework is response sensitive; however, the prescriptive mechanism 
docs not operate following each individual response, but only following a completed lesson, This means that 
(I) the lesson materials tl^cmselves do not need to be presented on-line» (2) the student has more 
opportunity to use media devices other than a computer temiinal, and (3) testing may occur ofMine with 
fomi-reader computer input following testing. 

A secondary role of the ii}gonthwlc model lies in t)ic pmchpt]on of counseling experiences. Three 
major contingencies can arise which may trigger this counseling role: (I) resource exhaustion, (2) 
timc-fatigue factors, and (3) situational peiformance factors. Certain kinds of counseling experiences are 
necessary as an integral part of optimal course How, These experiences include counsehng related to career 
goals and requirements, course incentive doscriptioiis, and acadenHc difflcullies caused by personal 
pfoblenis. These counseling experiences can be assi^^ied to fill a gap when the resources needed to continue 
inslmction are unavailable to the student. At certain times, such as after a number of hours on-task when a 
break is needed, son^c counsehVig may be assigned as a means of alteri(ig a task briefly. Whenever 
performance falls due to situational factors peculiar to the student, counseling may be assipied. The 
experiences suggested njay involve either an instructor, a professional counselor, or, in cases such as career 
counseling, a technological approach such as a slide-tape presentatiori, oi a CAl-counseling task may be 
assigned. 

Payoffs for AIS 

A number of potential payoffs accme to the use of a regression model within AIS. First, the 
operation of tiie model can be made transparent to (he learner. The interface between the student and the 
prescription of instructional materials begins witli a test on previously learned materials and ends witli the 
prescription of new materials. The student need be aware of only these two. components. Between them» 
the adaptive modefs (regression among them) operate without further input or interaction with either the 
student or instructor. In the rare case wherp specialized additional input is required, this is assigned or asked 
for, but the modePs operation remains transparent. 

A second potential payoff of regression models used in the prescription of learning experiences is the 
characteristic that assures near-optimal selection of available instructional software media and specialized 
carrels given proper predictive input. This characteristic can be validated withht the system to provide a 
check on the usefulness of the model. 
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The regression model wilt operate in real-time, providing near instant response to input so that when a 
student completes a diagnostic or criterion test, the prescription for the next activhy isavaitablc as soon as 
the results are fed into the computer (either through the facilities of a computer-based test or through 
optical mark reader input.) This will usually happen within a matter of minutes following completion of the 
test. 

Finally, the model may be updated in a cybernetic fashion In order to reflect changes in the relative 
predictive value of individua) predictors as a function of emerging changes in the characteristics of students, 
the relath^e quality of instructional materials, or other factors. 

Review of the Literature 

The literature bearing on regression modpls falls into two general categories, (1) applications of 
regression models to adaptive instruction, and (2) studies dealing with the selection of predictor variables 
for the regression equations. 

Applications. Three applications of regression models are described. Two related applications at 
Florida State University were described by Dick, Rivers, King, and Hansen (1970) and by Rivers (1972). 
Dick et (1970) investigated developmental procedures for producing a regression model to make 
decisions for ongoing instmctlon in a computer-assisted instructional task in Boolean algebra* Preview frame> 
criterion, acquisition, quiz performance, latency, and subjective confidence measures were investigated in 
order to determine the measures which would contribute most to a predictive and useful adaptive decision 
model. The prediction of qul^ score utilized performance^ latency^ and confidence measures on the 
criterion questions. Prediction of final examination score used performance, latency, and confidence 
measures on the quii questions. Prior to the time the student completed either the quiz or the final 
examii^ation, a prediction was made of his test performance. Remedial instruction was provided whenever 
the predicted test score fell below the mean established by a prior group of students. The results of this 
investigation were equivocal. In one unit, the adaptive model identified students who needed remedial 
instruction. Even thougli the students started instruction with scores below those of students in a reference 
group, scores on the final examination were comparable. A second unit did not produce usable results due 
to a mismatch between the diagnostic questions and the difficult training materials. 

Rivers (1972) investigated the relationship between within-cour^e variables and achievement in the 
course. Correlational and regression analyses were used to identify the relevant predictor variables and 
remediation points. During the first phase of the two-phase study, 33 female subjects were administered 
trait and state-anxlely inventories (STAl, Spiclberger, 1969) and then proceed through a program in health 
education describing the incidence and risk of contracting heart disease and the diagnosis of myocardial 
infarction. During learning, percent correct answers and response latencies were obtained for each student, 
and regression equations were computed from these data. 

During the second phase, cumulative performance and latency, as well as the most recent 
performance and latency, were used as predictors of the posttest score for each of the nine Concepts 
involved in the program. Whenever the predicted score indicated performance would fall below the 
acceptable level of 80 percent, instructic«^ was prescribed, A total of 80 female students were used to 
validate the regression models. The results indicated that use of the regression model led to improved 
performance on the posttesl. A control group which received ren)ediation on all concept also improved, but 
the group took more time and did not score as high on the criterion test as the regression model group. It is 
clear from this research that the regression model provided a means for assuring that students who required 
remedial instruction were given it. the time savings shown, although small compared to the atl-remedtation 
group, would amount to a large savings over the duration of a course. It may be noted that the regression 
model group did not take significantly more lime than either a "student choice" or a *'no remediation" 
group taking the same instruction. 

Input variables. The input variables may be considered along the framework proposed by Cronbach 
(1967). Adaptation to individual differences may be characterized by the natures of the educational goals, 
the instruction provided to the student, and the modifications to the treatment in order to meet the 
students* needs. Table 2 briefly describes the framework which Cronbach proposed. The regression 
approach which FSU is pursuing is an elaboration of the framework for goals fixed within a course with 
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TABLE 2. 



PATTERNS OF EDUCATIONAL ADAPTATION TO 
INDIVIDUAL DIFFERENCES 



Educational 
goals 



Instructional 
treatment 



Possible modifications 
to meet individual needs 



Fixed 



Options 



Fixed within 
a course or 
program 



Fixed 



Fixed within 
an option 

Al tcrnatives 
provided 



la. Alter duration of schooling 
by sequential selection. 

lb. Train to criterion on any 
skill or topic, hence alter 
duration of instruction, 

2a. Determine for each student 
his prospective adult role 
and provide a curriculum 
preparing for that role. 

3a. Provide remedial adjuncts 
to fixed "main track" in- 
st^^uction. 

3b. Teach different pupils by 
different methods. 
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(From Cronbach (1967)). 

alternative instnictiona] treatments. Cronbach gives two dimensions of modifications The flrst dimension, 
and that along which Dick era/., (1970) and Rivers (1972) have operated, provides for remedial alternatives 
to an otherwise fixed instructional program* Cronbach devotes considerable attention to the description of 
the adaptation of instruction to individual differences by altering the instructional method utilized (3b. in 
Table I). The investigation of aptitude X treatment interactions (ATI) is directly related to this application. 
Where ATI's are shown, differential instructional treatments may be administered to students who differ in 
the related ability, aptitude, achievement, personality, and affective characteristics. 

Aptitude X treatment interactions have been discussed by Bracht(1970) who analyzed 90 studies, 
only S were shown to produce significant disordinal interactions. Bracht defmes a disordinal interaction as 
one In which the treatment differences at two or more levels of the independent variable are both 
significantly nonzero and different in algebraic sign* In other words, the lines on the graph of the means 
must cross. Indicating that learning treatments are differentially effective at different points of the 
individual difference continuum. The four research studies which resulted in significant disordinal 
interactions and which are related to the present report may be summarized as follows: 

(1) Atkinson and Reitman (1956), found that students low on affiliation motive performed better 
with an achievement-orientation treatment and students high on affiliation motive performed 
better with a multi-incentive treatment. 

(2) In an experiment by Marshall (1969) students from poor educational environrncnts performed 
better on a high-interest task whereas students from good educational environments performed 
better on the low-interest task 

(3) Thompson and Hunnicut (1944) reported that introverts obtained higher cancellation scores 
when they received praise, while extroverts obtained higher cancellation scores when they 
received blame. 



(4) 



Van De Riet (1964) found that underachievers performed better when they received criticism 
and nomial achievers performed better when they either received praise or were asked unrelated 
questions. 
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The variables indicated above would appear to be fruitful input for an instructional model within the 
present context. T6 summarize, they are (a) affiliation motive , (b) prior educational cnviionment, (c) 
introversion-extroversion, and (d) prior achievement versus potential levels. 

Additional research by Snow» Tiffin, and Selbert (1965), 0*Neil (1970), and Tobias (1969) suggest 
that other variables such as anxiety level, attitude, and creativity might relate to instructional methods. Less 
stringent constraints on the selection process would atlow ordinal interactions to be utilized. Some of these 
relationswHI be described briefly in the following paragraphs. ^ 

Sriow, Tiffin, and Seibert (1965) investigated the effects of differential learner characteristics and 
prior knowledge ton immediate and delayed recall performance from both filmed and live physic lecture 
demonstrations. In an analysis of immediate rec^l, 225 *Ttlm*' students and 212 *1ive" students were used. 
Due to factors such as dropouts, a smaller number was used iji the analysis of long-term retention. No 
overall differences in performance leveh were found, but a significant difference in the amount learned was 
shown for the live-presentation group among students holding negative opinions about films. The authors 
concluded that passive observers lacking self-conftdence performed higher, but not significantly so, on films. 
Those high in numerical aptitude having no previous knowledge performed better In a live mode. A 
tendency was observed for low-aptitude students with prior knowledge of the subject matter to perform 
better in the film condition. 

The effects o stress on state anxiety and performance were investigated within a cc^nputer-assSsted 
instruction task by O'Neil (1970). Stress was induced by caustic feedback concerning performance. Trait 
anxiety and the number of errors v^tHn the task showed no relationship to stress. However^ state anxiety 
levels were shown to be related to the number of errors within the task. Students with high state-anxiely 
scores made more errors during the CAI task than students with low state-anxiety scores. However, the 
difference in number of errors was significant only in the easier portions of the task. 

In an investigation within a programmed instructional text, Tobias (1%9) found creative students 
had higher performance scores regardless of whether a constructed response or a reading mode was used, 
bu t that achievement was higher on technical material (in contrast to familiar material) for the constructed 
response group. Large differences vvere indicated between reading and constructed respcmse conditions for 
low-creatiyily students, particularly on technical pictorial subject matter. 

Model Structure 

A regression approach to adapting instruction will be developed which is, in part, based on liie 
research cited in the preceding paragraphs, A generalized flow diagram is presented in Figure 11. linear 
regression techniques with the least-squares criterion will be used to predict optimal uses of instructional 
resources by individual students. The instructional resources mclude the following: 

• Media options 

• Difficulty level 

• Redundancy level 

• Remediation 

• Content and sequence 

• fn^^mctional method 

• Hands-on/laboratory tasks 

The algorithm is the critical element in the construction of the student's learning prescription 
consisting of the previously lir.ted ou^ut variables. As distinct from the adaptive models to be considered 
which provide for within-task decision structures, the algorithmic regression model under consideration 
here may be thought of as a between-task model, making possible a more complex decision structure. 

The algorithmic model will operate in real-time, providing learning prescriptions based on 
up-lo-the-moment assessment of the student's history and current learning progress. For this reason, both 
trait or entrance variaWes, as well as state or concurrent variables, will be used in various combinations as 
independent variable^ in the prediction equations. 
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Fig. 11. Generalized Flow Diagram of Adaptive Regression Model Operation. 
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As a preJiminary starting point, the following items have been identified iS potential urdt variables: 

• AQE score 

• Mechanical aptitude scores y 

• Electrical aptitude scores 

• Verbal aptitude scores 

• Trait anxiety (STAl) scores 

• Trait motivation scores 

• Trait curiosity (OTIM) scores 

• General confidence measures 

• Opinions related to school, media, the nitluary, and the career field 

• Prior knowledge of course material. ^ 

State indices^ ie,., those which represent current and changing indiC'^s of the student's Icaming 
paramefofs, have been iden tifiec' as follows: 

• Recent performance scores 

• Errors on last unit 

• Confidence level 

• Mean response latency 

• State anxiety (STAi) scores 

• State curiosity scores 

• State motivation scores 

• Student's choice of future course flow. 

The input variables described will be utilized during a validation phase and the Islt wfli be amended as 
indicated by the results of the validation. The literature reviewed in Section IV indicates that these variables 
are related to learning in such a way as they can pie licl a learning level, or the variables are readily avaUable 
(such as AQE) and can reduce the error variance irl erent in the prediction eqj :::ons. 

Recommendat'ions 

Fwe recommendations concerning the use of regression equations for adapting instruction emerge 
from an analysis of the method and its feasibility. These recommendations are as fellows: 

• Bruner (1966) has made the distinction between descriptive theories of learning and 
prescriptive theory of instruction. The regression model appro?vh bridges the gap between 
description and prescription by utilizing descriptive measures vA xh have been shown to be 
predict^e. A prescriptive model is thus derived from a descrii, ti'- one. For this reason, the 
regression approach is recommended for scleclin^ instructional prefcription^ in MS. 

• Second, the regression approach appeals to be especially useful in an interlesson context. 
Rather than changing or modifying the instructional method with! i an instniccional session ou 
the basis of responses,a potentially n^*>r? effective instruclic: \ change caa be mude by 
directing the student to another set of materials within the lesson ^ 

lesson. 

• A third recommendaiioii concerning Uu^ v; of regression inoi ls focuses v\\ the interface 
between this and other .lodels with ihc . >;vess uf allocation >^ jnslruvij.mal software ami 
hardware. The interface itself has two :S^n>cnsions. The first dli.i^nsion rehles to selecting 
resources that match an nl^al prescription. A tnblc -look-up procedi re of some kind may suffice 
for this activity. The se.^ond dimension ^e?! ^vith the assignment of 'csource? ih it. may o: :noy 

O 
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not be ayaDable due to use by other students or within a repair/replacement cycle. ^Thls 
dynamic allocation procedure must exist in conjunction with, yet in addition to» the present 
Instructional model. 

It can be expected within an operable training system used to capacity that the prescription 
selected by the regression equation as optimal may be unavailable. A necessary Interface to the 
regression model formulation is a resource allocation mode! which would provide a list of 
available resources and estimates of resource use so as to permit optimal resource-spares to be 
ordered. Thus, not only will more efficient use be made of available resources, but resources 
can be maintained at an efficient level of supply as well. It is. therefore, recommended that a 
resource allocation model be developed as a part of AIS. 

As a fifth recommendation, validation of the regression models should be carried out over a 
sufficient period of time by the AIS contractor in order to provide sufilcient data to update the 
beta wei^ts, if required. The contractor should also devise a procedure for periodic 
revalidation and restructuring of the models. 



VII. DYNAMIC PROGRAMMING 

Characteristics of Dynamic Programming Models 

Dynamic programming, developed by Bellman (1957) and his colleagues at the Rand Corporation, 
grew out of a need for optimization in the war effort in the 1 940's, Dynamic programming continued In the 
1950*s in areas of industrial and other institutional problems. Operations research, as this scientific 
approach to the solution of industrial problems has become known, incorporates the formulation and 
application of mathematical models of optimization to the solution of instructional problems. Extensive 
applications of the techniques of dynamic programming have been made in inventory theory, allocation 
problems, control theory, search theory, and chemical engineering design. Because many of the principles 
Involved in Industrial problems also apply In the field of technical training, the techniques of dynamic 
programming can be utilized in the solution of allocation problems In an adaptive model in the AIS setting, 
or extended to become the AIS master model by monitoring and controlling other instructiorial 
subsystems. 

Basically, dynamic programming takes a sequential or multistage decision process containing many 
interdependent variables and converts it into a series of single-stage problems, each containing only a few 
variables. Bellman's dynamic programmming principle of optimality stales, **an optimal set of decisions has 
the property that whatever the first decision is, the remaining decision must be optimal with respect to the 
outcome which results from the first decision" (Bellman, 1957). Since dynamic programming essentially 
looks at a problem having N decision variables as N subproblems, the development of the high-speed 
computer has facilitated the high volume of computation that is required for an optimal solution. 

Dynamfc Programming In the AIS 

Dynamic programming is a technkjue that holds great promise in the area of decision-oriented 
systems in education. Such techniques allow for realistic modeling of the educational decision-making 
process. Dynamte programming techi Iques based on mathematical models are attractive as planning devices 
because they permit the specification of certain original states of a system, certain desired terminal 
conditions or targets, and then a search for the levels of system controls which will produce the desired end 
result within the constraints of cost and time, as predetermined by the overall system in which the 
instruction takes place. 

Since the Air Force technical training is complex system composed of many variables, a modeling 
process should produce results that are more accurate than mere intuitive or even empirically based 
heuristic decision processes, fn considering the mass of students, tlie vast requirements of the Air Force 
technology, and the considerable expense that is necessary to perform the kind of technical training 
required, it is obvious that a sophisticated mathematical model could produce decisions that would ensure 
greater efficiency of the training. The allocation of instructional resources, if handled in a dynamic and 
responsive manner would utifize the available resources maximally and provide feedback data regarding the 
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necessity of adUilional resources. If dynamic progranuning techniques were used simply in the area of 
Allocation of iiistmctional resources, n^axlnium usage of each resource, with minimizing cost, would 
result. 

Another concern of any technical training institution is that of optimization of time requirements for 
the stuileni lo attain the desired criterion performance?;. By employing dynamic programming techniques to 
monilof time requirements of students* progress through any instnictional sequence, more effective 
remedhl Dnd branching techniques could be used. Utilizing dynamic programming techniques would ensure 
that all studenis would complete the instructional sequence in optimal time so as to minLniizc costs while 
maxunizing instructional return. Dynamic programming techniques could also maximize time by specifying 
the level of criterion performance a student must meet in each instructional sequence before he is passed to 
the next one. A criterion level based on the results of a dynamic fomiulation could be optimized relative to 
both the individual student and the whole system, thus saying time and cost. This minimum criterion level 
would ensure satisfactory performance of] both the ppsttesl and in the job setting. 

A wider, more encompassing use of dynamic programming would be to monitor the complete 
technical training system. A dynamic programmmg forihulatton for the decision problems could be 
developed that accounts for the cost of instruction, rhe gain in competencies as a resMlt of the instruction 
(setting appropriate criteria), and the assigncment of trainees to appropriate training sequences based on 
differential student characteristics and other variables which are discovered to influence the training 
attetnpt. A model could be developed wh^ch would control and coordinate all other adaptive models and 
instructional subsystems. By utilizing proven existing systems, or usable proposed systems and/or 
procedure* within these systems, the dynamic programming model can optimize instruction based on stated 
goals and existing constraints without a major reworking of Instructional strategies. 

The ultimate use of dynamic programming technicques in all phases of adaptive Instruction requires 
thorough examination of all existing and proposed instructional alternatives (strategies and resources), all 
possible student characteristics and learning styles, and the goals of instruction, both desired and 
mandatoiy. This major effort should produce specific quantifiable variables which then could be submitted 
to actual testing in the inUructional situation. These experimental results would produce data on which 
optimizalioti decision processes could be based. 

An additional application of dynamic programming in the AIS setting could dynamically control the 
scheduling of students and courses so as to use the training resources and Air Force personnel most 
effectively. An individualized instructional system implies that trainees will be completing training at 
different times. To prevent nonuse of training resources, a scheduling procedure responsive to both the 
internal mstxuctional system and tlie external Air Force needs could optimize the scheduling of individual 
trainee pCi^ojinel and courses, 

Uteratute Review 

A description of recent uses of dynamic programming techniques provides a clearer perspective on 
their applicabilily to AIS. As previously indicated, operations research, to which dynamic programming has 
contributed heavily, can aid in solving industrial problems. Problems such as those of choosing the proper 
routes for roads, while optimizing traffic flow and minimizing cost, are discussed by Kaufman (1967). 
Other applics^tions of dynamic programming in decision-making situations include distributibn of 
investments to provide maximum profit, management of warehouse stock, and calculation of optimal 
storage capacity. 

Instnictional dynamic programmming techniques having greater relevance to AiS include those of 
Atkinson and Paulson (1970). Lorton, as reported by Atkinson and Paulson, used these techniques to 
examine an approach to the optimization of instruction in teaching spelling words toelementaiy school 
chddren so as to maximize performance on a posttest. As described by the Atkinson and Paulson article, 
the decision problem was lo find a choice strategy for achieving the criterion level of performance in (he 
shortest time* Depending upon the model of learning process applied, a method of selecting words to be 
presented in sublists so as to optimize posttest performance and minimize study lime was determined. The 
method of determining the selection procedure was dynamic in that the history of both the individual and 
prior students aided in defining the solution. The optimal selection method was shown in each case tobe 
superior to a fixed oMer of presenlaUon of words. 



Calfcc (1969) describes an application of dynamic programmming in determining an optima! 
procedure for presenting paired-associates to obtain criterion performance, while miriimizing the number of 
presentations, [n a similar application, Smallwood (197t) considers the cost for instruction as well as the 
cost of terminating the instruction prior to the learned state. Smallwood's paper develops a dynamic 
programrning formulation for deterntlning the optimum policy for presenting a multiitem list of 
paired-associate items. This optimum policy for the presentation of the multi-item list can be calculated by 
considering each item individually and then aggregating the Individual results into a total policy. Smallwood 
suggests that the applications of this method of determining optimal instructional presentation are 
particularly useful ih computer-directed teacher systems that incorporate a sophisticated decision-making 
capabOity. He suggests that the complicated computation needed for the initial determination of the 
optimal teaching policy can be done for many students simultaneously. Then as a new student enters the 
instructional program, the simple calculations required to adapt the instructional alternative to the student 
for optimal performance can be quickly performed, producing an instantaneous prescription for the 
student. PferiodicaUy, as more data become available from students having completed the instructional 
sequence, a complete recalculation of the optimum policy for each item and for the multi-item list could be 
carried out during off-teaching hours, when the teaching demands of the computer system are relieved. 
Smallwood suggests that this method of handling adaptation of instruction by computer may result In a 
more economical system, since the system would now be used more efficiently by trading off relatively 
inexpensive off-duty computation time for an increased student-handling capabflity . 

Calfee suggests that a simple mathematical model which approximates the learning process is more 
efficient than a complicated one. This is so because the mathematical analysis rapidly becomes more 
difficult as the mathematical model becomes more complex. On the basis of the derivation of an optimal 
presentation technique based on backward induction, Calfee concludes that presenting the item which 
yielded the maximum immediate gain was an optimal strategy. 

Kendrick (1966) suggests that a dynamic programming model could be used effectively for 
educational planning. He indicates that control theory models are attractive as planning devices because 
they enable one to specify certain original states of a system, certain desired terminal conditions or targets, 
and then to search for the levels of the system controls which will produce the desired end result at the 
minimum cost. He applied the model to determine the number of new students to be admitted to the first 
level of an educational system every year in order to meet the target levels of the desired number of 
students completing each level in the system in some future year.* Cost factors were also integrated into the 
model. 

Bellman et al., (1966) employed dynamic programming techniques to determine the pathways a 
psychiatric interviewing situation would take, with the computer simulating a patient. By simulating a 
psychiatric interview, a multistaged decision process built into the computer can, tlirough optimization 
procedures, become an adaptive decision process. By building a tree of questions and answers, each with 
certain assigned probabilities of occurring within the first psychiatric interview, Bellman and his group were 
able to present a realistic simulation of psychiatric interviewing for training purposes. 

Woods and Hartley (1971) report an iterative technique to determine optimum task difficulty ii)^ 
simple computational skills instruction. The goal for the computer-based instructional sequence was for the 
computer to generate a task sequence by varying the number of rows and the average number size in 
addition problems. In this way, any particular pupil at any competence level works with any specified 
probability of success. To accomplish this task, the computer continually re^estimatcs probability of actual 
success, generates examples, provides appropriate feedback to the model, makes a continuous error analysis, 
and stores records of student's progress. The process is dynamic in that decisions of task difficulty are based 
on the student*s past history and present status^ the history of other students, and changing goals (that of 
probability of success). 

To provide a basis for the initial stage of the implementation of a dynamic programming model, a 
suggested list of variables to be considered is presented in Table 3. The student characteristic variables are 
each presented with the interacting instnjctional variable as shown in at least one aptitude-treatment- 
interaction research study. 
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TABLE 3. 

Student Charactcris ticf> and Task Characteristics 



learning and Types 



Short-term nKimory 

Scienti fic interest* 
anxiety levoU in tro- 
ve rs i on/c X trove rs i on 



IQ, leading 
comprehension 



Associative [Vniory, 
induction, Coneral 
)X2a$oning 



General reasoning 

General reasoning, 
Anx'ieiy 

Anxiety, Active/passive 
learners 

Aiixicty 

Anxiety 



Anxiety 



1 ns true ti orial '"'Us k 
Characteristics 



Researcher 



PI step si ze 

Inductive/deductive, 
meaningful/arbi trary 

Massed/distributed 
practice (spelling) 

Overt/ correct correc- 
tion response 

Pacing, prompting 



Placement of review 

Exposi tory/di scovery 
(rule application) 

Example/negative 
cxarr^ple (concept 
learning) 

Expository/discovery 
Expos i tory/di scovery 

Sequencing 

Feedback » stress 

Test i tern difficulty 
Sequencing 

Test item di fficulty 
sequencing 



Furukawa , 1968 

Tallmadge & 
Shearer, 1969 

Fishman , Keller, 
Atkinson, 1967 

Suppes, Ginsberg, 
1962 

Gropper, Kress, 
1965 

Gay, 1971 

Dunham & 
Bunderson, 1969 

Tennyson, 1971 



Merrill , 1970 

Merrill & Towle, 
1971 

Gallagher, 1971 



O'Nei], 1970 

Towlo & Morrill 
19/2 

Munz & Smouse , 
1968 
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TABLE 3, (Continued) 



Learniiuj and lypos 



1 nYt Vii c tTo Ti a 1* ' 7'a s k ' 
Chdractcristi cs 



Researcher 



CiGncrtil rcasornnci 
onxioty» dependonce/ 
indopcnclcnce 

General rcn-^soning 
anxiety, dependence/ 
independence . 

Cenernl reasoning 
jnxioty, dependence/ 
Independence-' 

n Achievenient , 
n Affiliation 

Opinion, educi,tional 
level ;| 

SESj parent education 

in trove rs ion/extroversion 

level of achieveniont 

Inductive ! deductive 
"oasoning 

Aruiety (AAT ) , Sex 



Exposi tory/discovery Morri 11 , Kalisch , 
rules I objectives Towie, Steve, 1972 



Exposi tory/di5;covery Kalisch , Merri II 
rules, objectives Towlc, Steve, 1972 



Exposi tory/discovery 
rules , objectives 
student/program control 

Incentive 



Type of argumentative 
program 

high/low interest 
Blame/praise feedback 



Praise/reproof /neutral 
feedback 

Verbal inductive/ 
verbal deductive 

Constructed response/ 
no reinforcenK^nt/ 
reading 



Towle, Steve, 
Merrill , Kalisch, 
1972 

Atkinson & Reitman, 
1956 

Hovland, Luinsdaine 
Sheffield, 1949 

Marshall, 1969 

Thompson & 
Hunnicutt, 1944 

Van Do Riet, 1964 



King, Roberts & 
Kropp, 1969 

Tobias & 
Abramson, 1969 
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Though the literature produces limited applications of dynamic programming to adaptive instruction, 
the applications cited indicate that dynamic programming does have wide applicability to U\e area. Coupled 
with a high-speed computer, dynamic programming formulation of adaptive instructional procedures and 
policies could be employed to serve thou^nds of students engaged in amny different instructional tasks. By 
utilizing optimizing techniques, such as those that dynamic programming provides, the value of any one 
instructional segment or procedure Is never in doubt. In addition, the optimizing procedures will prevent a 
situation of greater demand on the resources than can be handled at any one time. ' 

Variables 

As indicated in the preceeding paragraphs, a dynamic programming model can be employed to 
combine and coordinate other models and system. Dynamic programming |pproaches used within an 
adaptive instructional model must include all variables which are germane to the proposed submodels and 
sulwystems, the meteraction and flow of these variables in an instructional-course unit, and the allocation 
and monitoring of resources (including money and time). 

All variables which are to be included in adaptive-optimization procedures must (a) be quantifiable, 
and (b) have an underlying continuous distribution. Furthermore, this underlying distribution must be 
given a sufficient chance to be reflected in the data. The effectiveness of any adaptive model is dependent 
on the quantification of the wliole system and the sensitivity of the measuring instruments employed. 

This quantification is most difficult in the area of goals and goal statements. Not only must minimal 
and maximal values be specified for any output, but the relationship of any variable to all other variables, at 
all possible values of all variables, must be interpretable at each decision ^oint. The problem of visualizing 
and following all these variables and their interactions so as to formulate an optimal solution (in terms of 
goal statements) is beyond human capacity. Therefore, the c6^ceptualizatlon of the problem (both the 
delineation of the variables and the interpretation of these into statements) is (Mathematical in nature and 
necessarily computcr*ba$ed. 

The decision about whether to use a dynamic programming approach at all is a relatively simple 
question Compared to the ojae which deals with the degree to which it should be used, and the energy and 
funds avaiable. After extensive research and planning, a dynamic programming approach can be divided into 
a number of categories, with each category implying differing amounts of commitment and implementation 
at different times. The nature of each categoiy, and what payoffs can be expected from each, is looked at 
in the following breakdown. The categories difier in<l) the sequence that are first dealt with, (2) the 
amount of initial commitment of each, (3) the amount of payoff that can be expected from each, and (4) 
the implications of exclusion and inclusions of differing combinations of each. 

An explanation ?nd an elaboration of the categories and functions outlined in Table 4 and Figure 12 
ftilow: < 

Function L Generating the initial adaptive model-including the decision model. Because this initial 
procedure is germane to the development of any system and Its formulation affects the choice of 
latter functions of dynamic programming, this implementation function is discussed in detail in a 
fdlowing section. 

Function 2. Updating algorithms as new performance schedules become available. Updating an 
algorithm is the fiisl part of a two-step process. The regression equation will first yield a more 
■ accurate prediction of the dependent measure (e.g., scores on Criterion Test No. 4). It will then be 
decided how this improved prediction affects the goals of the system (of which a specific 
algorithnvrcgression equation Is but a small part). The improved prediction from an updated 
algorithm may ultimately lead to better masteiy for all students (accuracy goal optimized), less time 
to a mastery level for most student (speed goal optimized), or elimination of some remediation loops, 
instructional alternative pathways, decision points, etc. (money or resource allocation goals 
optimized). The actual decision will be reached from the set of recursive equations derived earlier 
from the goal statements. 
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TABLE A. 

Categories and Functions of Dynamic Programming Approaches 



Category 



Function 



1. Implementation 



Generating the initial adaptive model 



2. Updating & Refining 



Updating algorithms (regression equations) 
as new performance schedules become 
available. 



3. Incorporation 



Incorporating new variables into the 
preexisting algorithms (regression equations) 



4. Adapting & Adjusting 



Changing the model as a function of changing 
goals (priorities) and major ^^source 
allocation changes. 



5. Scheduling 



Generating a formulation which optimally 
estimates the frequency one should use the 
updating and refining (Function 2). 
And how often one should attempt to 
incorporate new variables into preexisting 
algorithms (Function 3), 
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Function 3. Incorporating new variables into pre-existing algorithms. This feature may be a resul t of 

1° '^f u neiv research frndings. or new ideas from empirical data generated - 

by the systen^^itself. Most typically, a new variable {e.g. , a premeasure or a criterion test score) will be 
included m the existing regression equation. Data compflation and analysis wUl then follow to 
determine if the variable s inclusion can contribute to optimizing the achievement of present goals. 
Un Ike the updating algorithm stage, this stage is heuristic in nature and parallels the process of 
settmgup an original algorithm.* y 

nllV^ 4 /fte model as a function of changing goals (priorities) and maior resource 

ser OS of pathways, then each vertex is a point at which a decision is to be made in assienina a student 

each mstructional block. What is suggested here is a change in the number of dLision points as a 

^JdrnumbeT^f trctoi" "h""" ''t^^ ^^^^ -source^s'famrn o 

mdents "ist^ctors, phasmg out of equipment), and general level of competence of 

_ The repercussions of such major changes can be dichotomized in the following way f 1) 
disregard the model totally and begin again at Function 1 or (2) build the system so as to allow 
smiting goal statements and permanent shifts in student and aUocation variables. This second 

approach IS the only sensible, viable one for two reasons. First, any model which costs so much t6 
concepftjalize and set in motion cannot be scrapped because of minor external fluchjations. In a 
sense, the system must be externally dynamic as well as intrinsically respcmsive to the variables fed 
regularly into the system. Second, optimization procedures can only be effective within a fairly stable 
fraiTiework of goals. Dynamic programming approximates, over time, an optimal solution. Therefore, 
II the stabilization of content and/or goals cannot be guaranteed or defined for an extended period 
attempting a dynamic programming solution to the adaptation of instruction (and the systems 
approach itselO is probably a wastelof time and money. • 

At this stage of conceptualization, it is difficult to forsee how the mode! can.be expanded (as in 
Function 3) or condensed by any other method except adding or subtracting one decison point (i.e., 
set of pathways) at a time and evaluating the consequent effects. One simple way to condense a 
system would be to delete the least predictive variables and the least efficient pathways. Such a 
process would be relatively straightforward if the formation of the pathway system was a step-wise, 
iterative process. Because the cost in administering a scale and including a test or premeasure score in 
a regression equation is small compared to developing and maintaining different types of media and 
different difficulty and redundancy levels within a medium, many less efficient pathways would be 
deleted before any predictive variables would be dropped from the regression equations. 

Function 5. Generating a formulation which optimally estimates Jlfe ftequency that one should 
update the beta wei^ts (Function 2} and the frequency that one shouU attempt to incorporate new 
variables into a pre-existing regression equation (Function 3). The need for the updating and the 
incorporatmg funclions will decrease as the model stabaizes(ie., stabilization of pathways, decision 
pomts, and regression equations). This stability wUl conceivably be reached as the number of persons 
going throu^ the model increases and the number of combinations of new variables and ideas is 
exhausted. The importance given to the role of this updating and incorporating schedule will for the 
most part be decided m the formulation of the general system (FuncUon 1). However, its role is also 
this^Eon ^' ^' ^' ""'^ '^^^^^^ ^"^»'ty of resources allocated specificaUy to 

, Structure of a Dynamic Programming Model 

n,nKi " ^ '<""«'"be«d that Bellman's dynamic programming principle of optimal solutions to 
problems emphasizes the mterdependence of alt decision points. The number , characteristics, and 
^ "^r'^r^^'^f " P*?'^ function of the goals. As it appears now, a goal statement 

wll be composed of statements of desired mastery levels, of cost-effectiveness of each of the pathways and ' 
^qJSatle Jermf'" ^"^ ^'^^ constraints). The goal statement and the whole system inu'st be 
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Until a training situation is identifled and a goal statement specified, developing an opUmal 
prescription Tor the student is impossible. In practice, a goal statement can be developed, and a network of 
contingencies built within those constraints and priorities. Building an optimal solution where there has 
been no clear definition of goals must take into account every possible goals. At the present, that capability 
is neither realistic nor desirable. 

Implementation of Dynamic Programming Within an Established Functioning Program, Table 5 shows 
the development of considerations to be employed in a dynamic programming approach to an existing 
program in which goals have been clearly defined. 

Implementation of Dynamic Programming Without an Established Functioning Program. Whereas 
oi$timizing established variables and a stable program is conceptually possible, the task of optimizing and 
developing a program (instructional course/unit) simultaneously given only the goal statements is not 
presently possible. How many decision points, what kind of variables, what are possible pathways, how do 
each of these pathways relate to the decision points, what research should be examlned-these are too many 
elements to analyze and optimize simultaneously^ This can be contrasted with the Air Force training 
programs and other established programs where the question k more one of finding the best algorithms at 
each of the established decision points, with each of the available premeasures and criterion measures, and 
with each of instructional alternatives known to be efficient, % 

The only realistic approach at present is a simplistic, step-wise, iterative process; one in which 
programs are first defined and implemented^ and then optimlzatJon techniques are used as the system take 
concrete form. There is a large difference between an optimal system and a system that has been optimized. 
Until a thorough understanding of dynamic programming techniques is gained, the specification of an 
optimal system in this setting is impossible. 

The dynamic programming model is unk^uety designed to (a) handle the monitoring of the system; 
(b) fcJlow each student as he traverses from each instructional unit to the next; and (c) interpret and 
predict a student*s progress at each decision point in terms of the previously chosen goal values. Figure 13 is 
detailed flowchart of what student pathways and decision points may be included In an instructional course 
unit. 

As the student enters the task c\)ndition, his cumulative student profile is available (1. Student Profile 
Bank). The task requirements, as Initially determined by the task analysis, and requisite student skills, 
determined from data of previous students, are matched with the data in this student*s profile bank (2. 
Basic Profile Requirements). If the shident's profile is found lacking some prerequisites, the appropriate 
remediation is administered (2A. Remedial). The stirdent^s success in this remedial program is ev^^uated 
(28. Evaluation), and this evaluative information is then used to update the student^s profile bank. The task 
requirements and the updated profile are then again compared. When the rppropriate match occurs, the 
optimal instructional prescription (3. Instructional Prescription Algorithms) determines the appropriate 
cost^effcctive^and-available instructional alternative (4. Available Cost Effective Instructional Alternatives). 
The student is then given the training according to the prescribed instructional algorithm (S. Training. The 
effects of this training are then evaluated (6. Evaluation). This evaluative information updates the student's 
profile bank and provides feedback of the degree of succe^ of the decisions that determined the initial 
prerequisite behaviors (2. Basic Profile Requirements) and the subsequent instructional prescriptions (3. 
Instructional Prescription Algorithm). Information from the evaluative stage also determines the 
appropriate remediation algorithm (7. Post-Training Remediation), if needed. If criterion is met, the next 
task in the sequence is assigned. 

Playoffs 

The purpose of dynamic programming is to optimize. Thus, any payoff expected from using dynamic 
programming will be in terms of the goals that a program designer wishes to optimize and is able to specify 
in quantifiable terms. 

The dynamic programming payoff issue involves (I) consideration of implementing a dynamic 
programming model for a program (feasibility); and(2) the degree of commitment available for investment, 
and the amount and kinds of payoffs to be expected (maximization). 
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TABLE 5. 



Stages in the DevolopTOnt of an Optimization Solution 
in a Preexisting Instructional-Course UnU 

L Conceptual Stage 
-including: 

a. identification and enumeration of variables and levels 
' of these variables 

b. selection of goals 

2. Inferential Stage 
-including 

a» collection of subject data from (1) any previous research which 
has included variables that appear in Stage 1; (2) courses 
on other programs which have collected or are now collecting 
data; and (3) pilot studies- 

b. compilation and analysis of data 

3. Representation and Equation State 

-inclusion of those variables and those pathways 
which have been moderately successful up to this point. 

4. Optimization Stage 

•optimization of those variables and pathways which 
were included in Stage 3 
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Being in a position to decide if dynamic programming is to be used requires a knowledge of what 
payoffs are to be expected and what kinds and amount of inputs are necessary. It is suggested that a 
profitable way to view this tradeoff is in terms of payoffs and inputs at each categorical-functional level 
(Tabic 4). 

Issues and Conclusk)ns 

The obvious expenditure of effort and funds to implement dynamic programming techniques at any 
level in the technical training system requires resolution of several issues before Implementation can be 
attempted. The issues cited in this section are not intended to include all those vAiich influence the 
applicability of dynamic programming to the adaptive instructional modelsj but rather are illustrative in 
nature. 

As dynamic programming is best utilized in a complex^multideclsion process system, the length and 
complexity of the instructional sequence will be an important factor in determining the feasibility of using 
a dynamic programming model to adapt instruction. If it is discovered experimentally that prediction of 
performance task on an instructional sequence 1$ based on the scores of one pretest interacting with two 
possible alternatives, dynamic programming would not be appropriate to control the assignment of 
alternative instruction to students. However, if many student variables are found to interact with several 
alternatives at several points in the instructional sequence, dynamic programming techniques may be 
appropriate to optimize the decision process. 

In addition, thrdegrei of fine-grained adaptation necessaiy to produce significantly different results 
in optimizing on certain prespecified goals will determine the extent of involvement in the 
controlling/decision-making of instruction by dynamic programming processes. If the decision involves 
chofce of simple dichotomies, dynamic programming probably is not efficient in terms of effort and cosl. 
However, decisions based on variables, each having an underlying continuum, and producing varying degrees 
of goal attainment, require techniques such as dynamic programming to optimize the decision-making 
process. 

A third issue that must be resolved prior to implementing a dynamic programming model is that of 
the stability of subject matter within the instructional sequence. Dynamic programming processes require 
iterative calculations of solutions to recursive equations based on data gathered over many students and, 
therefore, over time. The data collected must be of the same kind; that is, from the same test items, same 
instructional requirements, etc. Therefore, if the s^ibject matter is stM undergoing revision, data on which 
optimization procedures is based would not be from a replicated instructional sequence. 

A most important Issue to be resolved prior to implementing dynamic programming is that of 
quantifying all variables, including goals. As these optimization procedures require the solutions to sets of 
recursive equations, all variables In the equations must be in the form of mathematical quantities. It is this 
determination of a quantification scheme that initiates the ordering and assigning values to goals, which Is 
the basis for the solution of the optimization equations. Most other input variables will be in terms of a test 
score, or some other easily quantiHable characteristic wliich will readily fit into the recursive equations for 
solution. 

■ ■ . ^ 

Though other issues will be met, the last to be mentioned is that involving the updating of the 
decision process. As this updating and revision requires the solution of recursive equations based on data 
collected continually to provide decisions responsive to the system and personnel, the issue of how 
frequer iy and on wiiat basis will the updating be processed should be faced. The inclusion of new variables 
emerging from research versus the stability of tfie instructional decisions will have to be weighed and 
appropriate guidelines developed. Issues regarding the cost of the solution of new equations, possible 
changes in necessaiy resources, changing goals, changing constraints, etc. must be resolved, and provisions 
for the accommodation of such changes buHt.into the decision-making system. 
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The following recommendations are made: 

• Though the requirements of a dynamic programming model of adaptive instruction are 
demanding^ the benefits accuring from such a decision process appear to be great. The inclusion 
or exclusion of variables are based on empirical data which then can be examined in the tigh t of 
optimization of prespecified goals, Decisons in the form of prescriptions for instruction are also 
based on optimization of goals. The administrators of the training system are assured of 
decisions which will produce optimal results in student performance, system time, and overall 
cost. Therefore, it is recommended that the AIS project develop a dynamic programming 
model. This development wU! require extensive research and development efforts. 

• The development of the dynamic programming model should be phased as follows: 

• Initial system description 

• Sin^e variable optimization studies 

• Multivariate optimization studies 

• The emergining of cost-effective criteria and decision processes with the evolved dynamic 
programming model. 

• Each of the AIS course operations could profit from the application of a dynamic programming 
model. This is especially the case for PME and Weapons Mechanics in that the model would 
provide for optimal utBization of expensive high-fidelity simulators and trainers. 

• It is recommended that the dynamic programming model be both inclusive of other models and 
focus on the optimal utilization of AF traijiing resources. 



VIII. NATURAL LANGUAGE MODELS 

Natural Language Processing (NLP) research began in the early 1960's in an attempt to develop 
computer systems which could carry on conversations with people and answer inquiries. As reviewed by 
PSrotte (1971) and Simpions (1971), thes^ efforts have been fairly successful within a constrained dialogue 
enviromnent. More recently, as reviewed in the following paragraphs, investigations have been made to 
utilize NLP for training and counseling purposes. Table 6 provides characteristic examples of some possible 
interactions between the student and system witliin an AIS context. 

Characteristics of the NLP Models 

Several characteristics of NLP have become clear in the context of adaptive instruction. The first is 
lhat NLP usage Is inherently adaptive because the students* knowledge, niental processing, and language 
capabilities must be modeled and taken into account in order to achieve an appropriate system response. 
Second, the NLP task determines both what the student is expected to do and what he will do, since the 
task characteristics are embedded in the conversational dialogue (e.g., a new definition or a new 
interpretation are presented for both naturalness of dialogue and achievement of the task). Finally, the 
interactive dialogue occurs within successive contingent language statements that clarify and focus the 
mutual student-system understanding of and progress toward the training goal. Any adaptive model for 
computer-based natural language interaction must, therefore, account for these three characteristics. 

Semantic Memory Models. In order to allow for the characteristics of the students* knowledge and 
their mental and language competencies, the NLP model to be presented is based upon the concept of 
semantic memory that is, the model of meaning functions consisting of verbal representation of concepts, 
properties, and relationships within an organized scheme. For example, given a taxonomic scheme such as 
vehicles, the semantic org3nization within the system and human memory can be represted as stated In 
Figure 14. 
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TABLE 6. 
NLP Dialogue Examples 



c. Please state your name, course, and problem. 

S. Wallace Mechner, weapons mechanic co'jrse. I wanted to find out about 
what I can do after the course. 

C. Uv> you mean advancement in your Air Force career Wallace? 

S. Yes, like more schools and changing to another field. 

C. As you mJy know, the Air Force has one of the largest educational programs 
in the world with vocational/technical training, and completion or « 
advancement in both high school and college. In addition many speciality 
schools exist. If you have a particular area of study in mind we can 
talk about it. Of course, much of what can be available to you depends 
upon how well you do in your present assignment. I see by your course 
records that you are progressing very well. Keep it up. 

S. I enjoyed the basic electronics lessons and would like to get in that 
area^ 



C. 



(C = computer and S = student) 



C. Please state your name, course, and problem. 

S. Irving Mannix, Inventory Management. I completed all of fry work early 
so far Xq have extra leave time but now I need money and want to knowt 
if I can change my request to get proficiency pay. 

C. Well, as you may know from the orientation session not all sections 
have leave time or proficiency pay as advantages for early completion 
and the two are not directly interchangeable. However, it is not too 
late to change your request. I can tell you hew to do that if you ' 
wish and we can discuss the proficiency pay scale too. 

S. I want to discuss the pay scale. 

C. O.K. The following blocks have pay incentives in addition to the 
overal 1 course time. ... 



C. Please state your name, course, and problem. 

S. Peter M. Eiroy. Precison measuring equipment. I am having trouble 

with the block on vacuum tubes. 
C. I see that you have been successful up through the third unit in 

that block but failed the tests on solid state circuits in unit 4. 

Do you think you have had sufficient time to study for this? 
S. Yes, but I still don't understand it. 
C. How did you do on -the workbook? 

S. Most of the problems threw me and I had to ask others what they did. 
C. By "threw me" do you mean you didn't understand the question or how 

to arrive at the answer? 
S. Yes. 

C. Well, I would like you to take a little diagnostic examination. It 
is defining terTrts and will help to pinpoint your problem more. Sign 
on to 0ITES3 and them come back, to me. 
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vehicles 




Fig. 14. Semantic Memory Model. 



The semantic relationship for the concepts (names of classes), the concept properties, and the 
relationship between concepts is specified by this hierarchical organization, that is, it can be derived that all 
cars are vehicles, that Jeeps are blue or brown, and that Jeeps are cars. Given a setting \^ere the purpose is 
to instruct a student in this taxonomy, the interactive dialogue could be shaped in precisely this organized 
way* After accomplishment of the terminal objectives, the student would likely have this material organized 
within his memory in nearly identical structure* Therefore, the NLP systern would be modeling the 
student's memory structure and processes. 

Operational Processes. The NLP model as implemented in the computer represents processes 
essentialiy similar to a reader who is attempting to comprehend a passage or a listener who ss interacting 
with another human in a dyadic context. As the student inputs a verbal response (usually typed), the NLP 
performs a pattern-operation transformation on the input (Simmons, 1970). The fust step in the pattern 
operation consists of searching for a given language pattern in the input. This leads to IdentlHcation of key 
words, phrases, or sentences (sentence decomposition processes). Parsing techniques may be employed. 
Parsing is a syntactical analysis process by which the parts of speech are identiffed. When a key pattern is 
found, a response generation process associated with the key pattern is invoked so that part of theinputis 
grammatically combined with a partially composed sentence so as to output the sy$tem*s response sentence. 

To enhance the sc^histicatlon of NLP procedures, additional sentence decomposition elaborates 
would be implemented. First, the pattern-matching routines can be guided by semantically controlled 
parsing procedures. Secondly, the order of key words, phrases^ and sentences matching can be structured by 
the previously noted location in the model's semantic memory. This leads to hierarchical decomposition 
analysis by semantically contingent processes that facilitate response recognition and speed. 

R(rfe of the NLP Models in AIS 

NLP is an optimal technique for encouraging and eliciting verbal knowledge and judgment responses 
from student*. For this reason, NLP is an excellent potential tool for evoking and evaluating complex 
concepts, judgments, and critiques. These are especially useful in counseling applications, the primary NLP 
application proposed for AIS. 
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The five content areas of AIS counseling are (1) the incentive management process, (2) career 
aspirations, (3) high-frequency learning problems with solutions, (4) student critiques of instruction; and 
(5) special motivational training. Counseling within the incentive management process may consist of 
dialogues about interpretations and implications of incentive schedules as previously presented by narrative 
materials describing the incentive programs In eadi course. Career counseling may be provided on an 
interactive basis whereby the student queries a data base (semantic memory) which wfll respond with career 
ladders; for example, a student's future in the Air Force can be handled in this manner. As high-frequency 
learning problems axe identified within each of the courses, the student can be given an opportunity for 
naMral language dialogue concerning the nature of the problem and potential solutions found successful 
with other students. In addition to the usual student critique rating forms, a natural language dialogue 
concerning the student's perceptions of the strengths and weaknesses of various aspects of the course could 
be collected systematically within these NLP counseling sessions. Student-generated comments on both the 
courses and his training experience at Lowry Air Force Base could be collected in this manner. Finally, for 
those students in need of special motivational training (e.g., need achievement training), the couseting time 
can be utilized for the presentation and interaction of special motivational training materials. 

AIS Payoff 

By handling Jhe counseling process by computer and on an as-desired basis for each student, 6ne may 
expect that not only the usual initial orientation for such areas as career and incentive counseling may be 
shortened or deleted, but also that resource people (instructors) may be less burdened with standard, 
repetitive discussions with each student. 

As a second area of payoff for the AIS, each student will have immediate access to a resource for 
solving problems. They will not have to rely on peers, or wait until instructors or Air Force documents 
become available. This will result in a third payoff in that it can be anticipated that students will be less 
likely to experience mounting interference in learning caused by umesolved counseling needs. Students 
often spend unwarranted time thinking about such problems until their resolution. 

A fourth area of payoff to the students would be the enhanced problem-solving and Information 
acquisition due to informal characteristics supported by the use of natural language. These characteristics 
'reduce the formal and semiarliflcial setting for the counseling sessions which would be necessitated by a 
standard techniques. The "naUiral'* approach not only allows for faster, more efficient problem-solving and 
information acquisition, but it also provides for reduction in the restraints found in traditional counseling 
situations. Smith (1963), Evans and Miller (1 1969), and Cogswell and Estavan(1965) have found that more 
candor and honesty were evident in computer-based comment acquisition systems than in personal 
interviews. Therefore, NLP approaches to counseling and associated dialogue requirements have high payoff 
potential. 

Review of the Literature 

Attempts to develop NLP systems for educational use began in the late 1960's (Simmons, 1968; 
Feurzig, 1969;Taylor, 1968; Tiedeman, 1968; & Carbonel, 1970). Most of the efforts in this area focused 
upon the problem of general analysis of language to determine intent and meaning; Le,, to develc^ a 
computational model of verbal understanding. However, as work progres^d in instruction, testing, and 
counseling, the student and the system were dependent on the objectives and strategies of the task. All 
factors important to the success of the NALP systems in an educational setting had to be integrated into 
the system itself. 

More specifically, the e/nerging research problems became the fcJlowing: 

• How can an educ^ tor input to an NLP system the specification of decision-making for strategies 
and content of conversation when the student is convcr ing with freely constructed language? 

• How can the stuAmt's characteristic bo taken into ^nint so that his verba) replies may be 
properly analy2ed and responeded to fully? 

• Hqw can the educator control the converse tional flow on some systematic basis and evaluate 
the interaction for its effectjaccording to educational objectives? 

i 

n ' i 
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The literature review is intended to show how the use of a semantic memory as on NLP data can be 
utflfeed to address these Ihree problem areas. After a discussion of current projecis, the review turns to the 
empirical and conceptional basis for a semantically based NLP model. This includes the research on 
semantic information processing, hierarchical organization of inemory in both people and computers, the 
concept of subject matter maps as related to semantic models of n^mory, the use of student maps of 
memoiy« and the student expectancies In a natural language dialogue. All of these allow derivation of a 
model which allows specification and rationale for the NLP adaptive model. 

NLP. Three current projects are discussed In this section which are considered to represent the best 
approaches at the present time. The first of these is a system called Protosynthex III, This project was 
begun at System Development Corporation, and it is being dev^oped further at the University of Texas by 
R. F. Simmons (Simmons, 1970; Simmons & Burger, 1970). This effort is representative of the other two 
efforts to be discussed in that it depends upon a semantic network for its capability to converse. In 
Protosynthex III, this network is used to generate semantic equivalents to student inputs. The tutorial 
declson model for Protosynthex III has a correct answer to a question in its data base, which has been 
formulated by an educator to contain only the necessary and sufficient information to answer the question* 
This answer is called the Canonical Answer (CA). The Student Response (SR) is taken by the language 
processor and checked for semantic equivalence to the CA, This process allows for determlnati<»i of exact 
sufficiency, excess detafl. Insufficient information in the answer, and correctness of the answer. The exact 
model consist of a 5 X 4 matrbc in which one axis describes the degree to which a student's answer 
semantically matches exactly, partially, overly, not at all, or incorrectly, the answer in the data base. The 
other axis describes correctness, relevance, feedback type, and the decision pertaining to the answer. Given 
a case on the first axis, one may derive the proper information for the second axis. For example^ a student 
who ansers with totally incorrect information may be considered incorrect and irrelevant and given negative 
feedback with a remedial response. Thus, the tutorial decision allows for system re^onses to conform to 
the correctness and relevance of the student response. 

A second project, SCHOLAR, has been implemented at Bolt, Beranek and Newman, Inc. based upcm 
work of Jaime R. Carbonell (1970). SCHOLAR also uses a semantic information netework as its data base. 
In a directed graph tree strucnire, the information units point to other subsuming facts, properties, and 
relationships. The resulting hierarchy can be searched both in termsof meaning matching and relevancy by 
consideration of the depth of search; that is, if a student is asked a question, it is reasonable to anticipate 
his answer as Simmons has suggested for Protosynthex III. If, however, students ar<J allowed to ask 
questions, as in SCHOLAR* the relevant responses to their questions must be anticipated by rules which 
state the relevant level of search in the semantic (cee. This is one of the most important problem areas foe 
SCHOLAR. Another way of slating the problem is, "what is relevant as well as correct information for 
responding to a students query?'* For example, jf a mieat in the AIS context asked about IncenXlns for 
early completion of the Inventory Management course, it would be helpfuLto respoQd with '•added leave,'' 
but not with "p^'omotion." While promotion may be part of the incentive system, it would not necessarily 
be suggested for early completion of the cours. Therefore, it would be inappropriate as well as irrelevant to 
the query. The decison-making for SCHOLAR proceeds on the basis of weighted and arbitrary rules 
constrained by context rules, These context rules provide the order and string size utilized in the 
decomportsition process. 

The final project to be discussed is being developed at Florida State University. Recognizing the 
problems of decision mtes for relevancy and content, FSU staff members have been working to Implement 
a systematic method by which educational objectives could be formulated for NLP, and conversation flows 
could be controlled and prespecifled by sum&es of the course instructor. This eUoii has been based upon 
the work of QuilUaa(l967) and the current research concepts of human hierarchical memory organization. 
Basically, the technique requires one to specify an expected hierarchical organization of some subject 
matter throu^ which a sUident may proceed level by level, in level increments, and in ascending or 
descending order By task znTiyzSng the subject matter in terms of an expected memory organization, one 
hopefully wfll be able \0 stnicture the data base so as to restrict Irrelevancies and incorrectness of the 
dialogue. Thus, this is basically a systems analytic approach with a task analysis of expected memory 
organization. 
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Finally, while noVbascd on semantk networks con<^pU/one of the most appi(^riattNLPpiciecU is ' 
the Informalion System for Vocational Decisions (ISVD) which Is a c<miputer-based system for providing 
career guidance. The project Is a product of several members of the Harvard Graduate School of Education 
(Tiedeman, 1968; Ellis & Tiedeman, 1968). One of the goals of the project was to allow an inquirer to 
make career decisions by obtaining vocallonal information from the data base. The inquiry was to be in the 
inquirer^s natural language since this reflected his Internal world (Ellis, Pincus^A Yee, 1968). In 1969, the 
ISVD project added an additional dimension in that the system was to be studied for explicit instmctional 
usage (Roman, 1969). 

Semantic Networks and the Organization ofSttu^ent and System Memory. Semantic networks were 
Introduced as a basis for NLP by QuOlian (1966. 1967, 1969). A semanUc network Is essentially a data base 
of facts, concepts, and relationships which, by its hierarchical structure, defines and gives meaning to words 
in terms of other words. Quiliian uses the analogy of a dicUonary in which definitions contain words which, 
in turn, may be looked up to further delineate meaning. This process may continue until all words are 
accounted for In terms of other words, resulting in a hierarchy of concepts, properties, and relationships. 
Qumian considers such a model of meaning as useful not only in an NUP setting, but^lso as a model of 
human memory. CoUins and Quillian (1969) have, in fact, tested some of the assumptions of the 
hierarchical nature of the model and found them to apply to humans in a retrieval task* 

Mandler (1967, 1968) has also addressed the organization of human memory In terms of a 
hierarchical scheme. On the basis of Milter's (1956) concept of chunking, Mandler has concluded that 
people organize information classes in sizes of S ± 2 information units. Because of the limits on intake of 
information, people must chunk or reclassify; when too much information is available for input, subclasses 
or new chunks must be generated. This allows the following three principles to be stated: 

(1) Organization is a necessary condition for memory; 

(2) The organization of verbal material is hierarchical; and ^ 

(3) The storage capacity within any given category is limited, thus necessitating subcategories 
which are also limited. - 

Handler's studies have generated a great deal of research on how people organize verbal memory, 
Wortman and Greeberg (1971) further investigated verbal memory by looking at the recoding process 
performed by a student when given a specified organizational scheme. They found that a student must (1) 
perceive the specified organization of categories. (2) chunk within superordinale categories, and (3) 
establish the hierarchical relationships between superordinate and subordinate categories. This actually 
specifies the components of the organization process suggested by Mandler. 

• One of the issues discussed by Mandler is of particular importance for the context of the adaptive 
NLP model, namely* a specific organization which is helpful or hindering to a student. Mandler suggested 
two strategies for organizations: a priori experimenter stmctures as opposed to subjective organization. A 
student's subjectWe organized memory seems vastly superior. Bower (1969, 1970) has suggested a similar 
process in order to retrieve information from memory* Information may not be retrieved unless so 
organized. Given the storage limits on categories. Wood (1971) has investigated the effect of forcing 
reorganization of memory information units and found significant negative effects when forcing two or 
more category reorganizations. Therefore, the NLP model should allow for Individual organlzatiwial entries 
of categories and relationships whfle maintaining the necessary and sufficient relations among concepUasi 
viewed from training objectives. 

Subject Matter Maps as Semantic Networks. Seidel (1969* p.3) has suggested that the subject matter 
for any Instmctional use must be organized in a manner which will provide ''the decision model with a bads 
foi deciding which factor, principles, or procedures are built on which o Aer facts, principles, or procedures, 
what interrelationships exist among them, and which topics may be presented before or after ^ich other 
ones." Seidel suggests that, without such a map, an instnJcU<»ial decison model cannot tell where it should 
start, what the next step may be, and how to arrWe at a terminal point. This concept of a subject matter 
map is conceptually equivalent to an expected student memory network. The decision model would consist 
of the rules for Interaction between the subject map and student. Thus, the NLP model should consist of r 
both a subject matter map based on memory organization adn the rules of interaction for the map. 
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Student Maps for Semantic Networks, ft is o. vious from Ihc previous discussion that any form of 
NLP interaction is dependent upon the student orgrjii/ation of memory. This necessitates the recognition 
of a student classification and organization scheme, a student map and a description of the instructional 
flow during student Interaction with tlic system in..y oe used to highlight the similarities and differences 
between the student's organizational structure anci ;hatdesigned into the instructional program. This 
includes noting the lack of organization and absence of information within the student's memory. Schank 
(1971) has suggested that people inter;»ct in an NLP or human conversational setting, on the basis of several 
modes of expected context, ranging from societal norms to individual preferences. While Schrank*s 
expectation classifications are not pa/ticularJy operational, they do point out that a student map of 
interactive flow must take into acc< ' ^ different possible contexts or orientations of the student which 
mediate and determine the memoiy oiganization. The student map is that operational part of the model 
which provides for context and meaning. ^ 

Counseling Expectancies. BlU and ticdeman (1968) in the ISVD project have suggested that 
expectancies for students in a vocuiional counseiiag setting are based upon the early states of the 
dccision-makijig process, Ihis concerns the need to acquire information with which to clarify choices and 
to plan for alternatives. During this period, a student must relate the information to personal attitudes and 
feelings. This may be considered Schank's expectancy, and relates to a student map In that a given flow 
through an NLP interaction should provide a picture of the student's exploratory state and direction. In 
this early slate, the decision rules model must use ihe student map to determme the information needed by 
the student. 

The Counseling Process 

As viewed here the counseling process consists of j the task characteristics, the decisions, and the 
information-processing a student may utilize. 

Task Characteristics. Tasks in a counseling situation are seei> to have .two basi(i characteristics. The 
first is that of information dissemination. That is, given that a student is looking for incentives or career 
possibilities, he must be ^ven information regarding career choices and their contingencies. Incentive and 
career counseling are seen as being more directly dependent on ir)|ormation dissemination than the other 
forms of counseling discussed previously. The second mode consists of motivational presentation of an 
intrinsic nature that will affect the basic drive processes of the student {e.g^^ need achievement training). 
The information for these two types of counscUug modes should be tehtiveVy more organized and 
complete. Further, the subject matter has an inherent organization, both hierarchical and categorical, which 
lends itself rather well to subject matter mapping. 

Counseling for motivation and learning problems, on the other hand, may be more a matter of 
memory reorganization and coherence than pure dissemination. While information is being presented, the 
dialogue engenders motivational and learning problem conflicts with the ideJized learner as represented in 
the program. Thus, memory reorganization is undoubtedly critical. The student mapping and decision rules 
must reflect this expectancy on the part of the student. That is, the interaction will occur in the presence of 
a conflicting predisposition within the studet^t. 

V 

Decision-What and How to Ptecsnt. feellman (1971) in investigating interviewing techniques, has 
identified two basic types of decisions whichVe useful in the NLP context. The first of these is a one-stage 
process in which decisions are made on the basis of only the previous information unit. The more typical 
decision process is a multistage process in which a sequence of decisions over time or contextual sequences 
are involved. Time is seen as the prime paiamete^* 

The initial interaction and much of the information dissemination in the counseling process are 
viewed here as requiring one-stage decisions. For motivational and learning-problem tasks, multilevel 
decisions would involve the model's decision rules which associate a student map with an information unit 
in the content map. Further, the multistage process is essentially iterative and requires continuous 
adaptability over time in terms of student map. It is in the consideration of a multistage decision process 
using rules which associate student maps with content maps that the NLP model becomes cybernetic. 



86 



NLPModd Stnichire and Variables 

FJ^re 1 5 provides a schematic representation of the proposed model. Processes 1 and 2 represent the 
activities" which arc part of the instructional systems analysis for NLP as used by the Rorida State 
University project, described earlier. The result is a subject content map organized as a semantic network 
(Process 3) which is related to educational objectives. This network is the basic content for presentation to 
the student ^itch that the student may interact meaningfully (Process 4). After eliciting a student response, 
it must be analyzed to determine word and phrase classes (Process 5). This is done in order to allow 
determination of semantic equivalance and may necessitate use of the semantic network. No actual 
correctness, relevance, or meaning is attached to the response at this point. It is throu^ Process 6 that 
these determinations are made by the set of decision rules. The rules model also check prior history from 
the student map (Process 7) and chooses the next presentation according to the semantic network. At the 
same time, the student map is updated. 

This is the structure of the model. The three main determiners of the model are Processes 3, 6, and 7; 
subject content map, decision rules and student map, respectively. It is within these three general classes 
thai variables may be derived. 

Given the nature of the content maps, student maps, and the counseling process as discussed in the 
previous paragraph, the design variables Cor the model can be identified as content, student, and 
decision-oriented group clusters. 

Contend The oigarJzation of subject content maps must be based on the task. Three prime sources 
dictating dt|anlzation exist First available materials which may be utilized by students prior to NLP 
interactlpp >^have an existbg organizaticm and must be considered. Second, the emphasis on information 
units by the educational institution should be reflected in the defined objectives that effect the NLP 
organization (for the AIS, this is the ATC and Air Force, in general). Finally, the expectancies and 
predispositions of students must be utilized in order to optimally orgamze ccmtent. 

These, along with the actual subject content, may be combined in an analysis according to, 
hierarchical organization by concepts, properties, and interrelationships. These are the input variables with 
the output variables being Ufe content map. 

Student The student map generation differs from the content map in that it is developed during the 
interaction whereas the content map is developed prior to interaction. The student map acquires data on 
each student's flow through the content map* Infonnation must be acquired during this flow which 
, includes the following: the network modes passed through, the next mode chosen, the noting of missing or 
incorrect information units for the student, and the noting of inapprq)rlate student organization according 
to an objective. The student map thus provides the data for multistage decisions. 
" AfapFlowx The counseling process includes all of the decision rules which must be included in the' 
model to relate student m^s to content maps. The one-level decisions need only note the student map for 
the immediately preceding event. The multilevel decisions must be based on rules noting more ofthe 
student's map. The actual decisions made and the strate^es used are a function of the objectives, the 
content, and the students. For example, if a student inconectly re^onds to a question concerning the 
relationship of two concepts but has correctly used the concept nanoes previously, it would be desirable to 
establish that relationship for the student as the next step. If, however, the student has not gone through a 
flow which required the correct naming of one or more of the concepts or their properties, these must be 
the next steps according to the hierarchical organization principles. 

Recommendations 

Three areas of recommendations may be noted. These are relationship of the model to NLP systems, 
training for use, and validation. In order to implement such a system, the following three items must be 
considered: 

# Model Relationship to NLP Systems. The model has been developed wi^^e consideration that 
it would be desirable to embed it within current or future NLP sylmls. This god has been 
achieved to the extent that the model may be implemented within any NLP system regardless 



Er|c 87 



CO 




< O UJ 

K s: >-4 to 

^ r: CO 

3 S§ < 
:3 H iiJ H 

</> ?t CO < 

o 



CM 



>r O »^ 

CO CtC iO 
Ol >- 

o ^ a 

<C 

H 0< 
<^ 

ecu} in 
X o 



"TTs 



in 

53 



CO 

to ^ 
>- < a: 
.J crt o 

< 

uj > 
< <c 

u. tu 

</) O CO 



ERIC 



88 



of the techniques used In response analysis. Any given NLP system is represented In Figure 14 
by Piocess 5, response analysis. Two major orientations arc used currently in response analysis 
for NLP-syntactic and semantic analysis. The present trend favors a semantic analysis 
approach, although a few years ago, pure syntactic analysis was considered of prime 
importance. In fact, some current researchers are of the opinion that semantic analysis Is both 
necessary and sufficient (Hays, 1$67; Quillian, 1967)* If syntactic analysis is utlized, the result 
of the analysis is fed direcdy to the decision rules processor If semantic analysis is utilized, the 
semantic network should be embedded in the larger semantic data base, and the decision rules 
may then receive the results of the semantic analysis. Thus, the semantic analyzer would use the 
content map directly. 

• The authoring of NLP dialogues has not been sufficiently recognized as a problem iiwthe 
literature. It is nevertheless a problem directly relating to the specification and evaluation of 
NLP dialogues. The systematic approach of content mapping discussed earlier is an attempt 
toward solution of tfiis p/oblcm. Any educational Implementation of NLP on more than a 
research scale should provide training in this systematic approach. 

• A related problem is the validation of the dialogue system. Several possibilities have been 
suggested as a result of the work done at FSU. The first is to follow a standard systems 
approach relating instructional objectives to test questions, and, through the hierarchical 
content map, to categories or levels of the hierarchy, A pretest/posttest procedure would 
probably suffice for data collection. The second possibility is to use a sorting or categorization 
task as the testing situation within the same system approach. This would provide needed 
information concerning memory organization. 

• In reject to the AIS courses, it is recommended that the NLP model be implemented and 
evaluated, especially in the student evaluation at\d counseling requirements. 



IX. AUTOMATA MODELS FOR ADAPTIVE INSTRUCTION 

Characteristics of the Model 

The type of model described in this section introduces a theoretical orientation for describing and 
predicting the states of learners. This theory is not new, but, to date, it has been used primarily in 
disciplines other than the behavioral sciences. Specifically, it has been used in computer science and 
engineering to study the behavior of complex machine systems. The approach to be described is called 
Automata Theory. The conceptual basis of the theory allows its application to the study of human 
"systems.*' 

The word ^'systems" is cmcial to the theory* Automata theory is an approach to the modeling of 
systems. It Is part of a larger theory of systems and Is distinguished by Its logical and mathematical 
approach to the definition of systems. A detailed discussion of this basis for defining system parameters 
shall be provided later. Some of the models thus generated which may have relevance for training will also 
be discussed. In this section, only a brief characterization of this body of knowledge is developed; 

The basic assumption of general systems theory'^as applied in the engineering professions, is that any 
physical device can be represented functionally in terms of relations between three types of variables which 
describe input output, and the state of the device. This is the general paradigm for a system as illustrated in 
Figure 16. 



Device 

Input State Output 

Fig. 16. General Paradigm for a System, 
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The hope seen for adaptive modeling ulilUlng automata models j$ for the development of a technique 
that is sufficient to handle the many complex tasks which elude current instructional models. Automata 
theory, with its core requirement of slate representations, lends Itself reasonably well to such tasks. 
Consider, for example, the task of electronic troubleshooting. This capability is paramount for good 
electronic technicians and required in many technical courses and jobs in the Air Force environment. The 
criteria for what makes a good troubleshooter are elusive. That is, it is difficult to task analyze the situation 
to specify instructional objectives because the problem-solving sequences are not readily analyzed in terms 
of perfomiancc. One could, however, analyze the task in terms of possible problem-solving states by which 
a troubleshooter may be viewed. The representation of such states is, of course, the most difficult problem 
for automata theory and Is discussed further in later sections. The Important point to be made here is that 
the state analysis may make a representation of complex behavior more feasible than is currently possible. 
If the representation is realized, automata models provide a powerful tool for prediction within an adaptive 
system. 

The possible role of Automata Models in the AIS is suggested in the preceding paragraph; It is a 
futuristic role since much work remains to be done in the area of state representation before an 
operationatism can be realized. However, given the importa.nce of complex behaviors in many critical jobs 
in the Air Force environment, the possibility of a more systematic tool to provide more people with the 
required capabilily (less failures, less mediocrity, less training time, and, therefore, more on-the-job time) Is 
worth striving toward. In this regard, the payoff is possibly high, although a quantative payoff level cannot 
be presently ascertained* 

Literature Review 

Simon (1969) has recently stated the proposition that not orJy can man be conceptualized as a 
system, but that: '*A man, vii^wed as a behaving system, is quite simple* The apparent complexity of his 
behavior over time is largely a reflection of the complexity of the environment in which he finds himself/* 
Simon suggest the example of an ant who, while traveling from one potnt to another, appears to follow a 
complex path. However, this seemingly cwnplex behavior is not so much a function of the complexity of 
the ant as it is of the complexity of the environment through which the ant must travel, [t is suggested that 
the ant actually is operating with very few rules as it detours around hills and follows gullies. Just as a 
complex machine maybe described in terms of functional components and performance characteristics, so 
may man. It is possible to identify such functional components as memory, perception (Simon, l967a), or 
emotional control of information processes (Simon, 196?b), This is a way in which to characterize human 
systems for use in an Automata Model, It is the characterization of systems and their device states that is 
important to automata theory. This literature^ review is thus oriented toward schemes of system 
characterization which may be fruitful for training systems. 

Background. A naive definition of an automaton might be that it is any object which appears to be 
self-acting. However, this defmilion should only be used impresslonistically. An automaton, according to 
formal automata theory, is defined by a rigid lo^c to be discussed shortly . 

Formal automata theory was begun in the late 1940*s by a mathematician who noticed similarities 
between the behaviors and structures of men and machines. John Von Neumann (1959, 1966) over a period 
of several years developed a theory which he felt could model both types of systems. The basic paradigm 
for automata theory is the system as described earlier and represented in Figure 16. The processing 
mechanism between input arid output is assumed to be composed of storage and combinatorial logic 
elements. However, the actual physical structure is unknown, and the prime interest is in the .properties of 
the system and the manner in which it processes information. 

Because the system is assumed to have storage elements, the current output depends upon the history 
of the syslem. Both input and output may be represented as a sequence of symbols. The system remembers 
both part of the history of the inputs and responses; this history describes the current state of the system at 
any given observation. Thus, given the current input and state^pf the system, the output should be able to 
be predicted when the system is adequately described because the output di^pends on only these two 
factors. 
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Booth (1967) has characterized the critical problems of automata theory as falling in two categories. 
The first is that of system characterization. That is, given a system, the characteristics thai determine the 
system's overall behavior must be defined. Depending upon this characterization, any of the two dozen or 
so currently defined Automaton Models may be chosen. The second problem is that of signal 
characterization. That is, the information contained in the signal must be prescribed by some relevant 
combination of the signal properties. Most of the review to follow is concerned with two primary 
orientations which arc currently used to characterize automata within the behavioral sciences-S-R theory 
and information processing theory. In order to fully understand these theories in terms of automata 
modeling, a brief discussion of formal automata theory is necessary. Throughout the following text, the 
basic theme is to suggest appropriate ways of system and signal characterization. 

Automata Mathematical Basis. Although Von Neumann did not define the mathematical basis of 
automata, others have continued toward this end (Arbib, 1964, 1968). The mathematics of automata has 
taken the form of one of the newer areas in mathematics, That of abstract algebra. This algebra is based on 
set theoretical notions. One first defines the sets of elements (for instance input and output symbols) to be 
described. Given the set, the next step is to identify the logically acceptable operations and types of 
elements for that set. Depending upon the number and types of acceptable operations and the types of 
elements within the set, one may then identify an appropriate algebraic system which has been defined for 
that universe of sets. In this manner, the modeling proceeds according to the logic of set theory to the 
mathematics of abstract algebra. Figure 17 shows the hierarchical progression of algebraic systems which 
may be used corresponding to different types of sets. Most work In automata theory at present Seems to be 
done at the level of the semigroup. A semigroup has only one operation defined on a set. 

As an example, let us discuss an automata which may be' particular application in modehng human 
behavior, the pushdown store. A good analogy to the pushdown store (Hopcroft & UUman, 1969) Is the 
stack of plates on a si3ring often seen in cafeterias. The spring below the plates has just enough strength so 
that only one plate appears above the level ofthe counter. When the top plate is removed, the next dish 
pops up. If a plate is placed <m the top of the stack, the rest of the plates are pushed down. In the same 
manner, the pushdown store is conceptualized to work with symbols within a system. Symbols may be 
considered to be on a list. Only symbols at the top ofthe list may be removed, and symbols to be placed on 
the list must be entered at the top. When a symbd is entered, the other symbols on the list are pushed 
down. If the desired symbol to be accessed is, for example, tfee thifd down on the list, the first t>Vo symbcJs 
of the list must be removed. They are popped off the list to retrieve the desired symbol. \ 

More fomially, the pushdown automaton is conceptualized as in Figure 18. It consists of an Input 
tape of symbols which pass a read-only head in a sequential manner. There is also an output tape with a 
write-only head also operating sequentially. Connected to the processor ofthe system is a memory unit 
which is a pushdown store. Algebraically this system is a semigroup. 

Several variations of the pushdown stbfe may be noted. This type of automaton may be a transducer 
or acceptor. When used as an acceptor, (the input changes the state but generates no output), the 
automaton may be used in the study of languages. A pushdown transducer (the input is changed by the 
state to generate an output) used to study memory is described later in this report. The pusdown 
automaton may be deterministic q^ondeterministic. A nondeterministic model Is one which allows more 
than one output symbol for a giv^h jnput symbol. 

Let us further use the stack of plates to characterize the operation of the pushdown store. Suppose 
the plates are of different colors, say, red and blue. The system may be defined to have rules which 
stipulate that if a red plate is input and the system is in state then add a blue plate to the output to 
change the state (o q"^; if the system is already in state q^, it may remain in that state. Similar types of rules 
may be defined for other possible situations for variations of input and state combin ations. In this way, the 
system operation has been characterized insuch a way that the output for a given string of input symbols 
maybe predicted. 

The next such section describes the actual use of a pushdown automaton model for psychological 
processes. 




91 



Sets 
Set Operations 
Union 

Intersection 
Complomontatlon 



Partitions 
Helations 

Equivolencc Relations 



Lattice theory 
Boolean algebra 



Operations on elements 
Identity e1ef?ient 
Inverse eleiwnt 
Congruence relation 



Systems with 
one operation o 



r 



Semigro up _j 



Identity element 
introduced 

( Monoi'd ~~]— 



Every element 
has inverse 



J Free H 



Group 



Cosets 

Invariant subgroups 
Quotient groups 



0 conmitative 



Abel ian 
group 



Systems with 
two operations 
0 and n 



□ a closed associative 
operation introduced 

U and 0 are distributive 



Ideals 
Difference 



J'- S - with operation [] 
is a semigroup 



Integra] 


1 clo(fi':»in 








is 







l ioUl ; .;""f ini to^^'^^o^s field 

Mg. 17. Block Diagram Illustrating Interrelationships of Different Aleebralc 
y^. Concepts From Booth (1967). Aigeoraic 

92 



ERIC 



INPUT 
TAPE 



READ 
HEAD 




CONTROL 
UNIT 




JOP OF 
LIST 



ITEM 2 



ITEM 3 




PUSHDOWN 

WORKING 

MEMORY 



OUTPUT TAPE 



Fig. 18. The Pushdown Automata. 



Infoimation Processing Characterization of Automata 

The General Problem Solver (GPS) (Shaw. Simon, Newell & Ellis, 1958) represents a prototype of an 
information-processing representation of automata. While the model can be considred an automata in the 
Von Neumann sense of the word, it does not utilize the forma! automata theory mathematics. It is 
discussed here in the context of automata characteristics and the representation problem. 

The overall goal for GPS is development of a computer program of human problem-solving that is 
capable of working with k variety of problems. Generality is the major goal of CPS with the index of 
generality being the number of solvable problem types. Some of the types of problems successfully handled 
thus far include sentence parsing, calculus problems, and verbal problems. The kinds of problems attempted 
are simple by human standards, but they do require intellectual effort of more than a simplistic domain. 



The proWem-solving techniques of the GSP are based on the concept of purposive behavior, a coiicept 
much discussed by Von Neumann as an automata characteristic. The GPS problem-solving techniques are 
organized by goals. By this, one means that the main function of the proving-solving techniques is to 
achieve the problem solution or goals, to do this, subgpals are generated in order to reach main goals, A goal 
ss defined in the GPS program as a data structure that provides sufficient information to carry out 
problem-solving actWities, i.e., a goal defines a desired state of affairs, the current situation, and a history of 
previous attempts to achieve the goal. GPS has four types of goal routines currently, and the necessity for 
other types of goal routines has not arisen. These four types of goal routines are as follows: (I) Transform 
Object A into Object B, (2) Reduce difference between Object B and Object A, (3) Apply solution operator 
to Object A, and (4) Select the elements of set 5 which best fulfai a criterion C. 

For each of these goals, methods are generated to obtain the goal. The methods for a given goal are 
represented via a tree-structured list as shown in Figure 19. This tree is called a "discrimination net** 
because of contingent-sorting capabQity. The terminal nodes of the tree are the methods for a goal. The 
solution selection is performed by discriminating, first at the top node, and then at each node in turn until 
arriving at a matching node. The learning process in GPS is the growth of these tree structures. 

Just as methods for goal routines are represented by trees, the representation of a real object or a 
concept is also in terms of trees. An object or concept may be any unit in the problem domain which we 
wish to utilize (i.e., an information unit). The major process of GPS in its attempt to solve problems is tree 
searching, which is a process of sequentially checking for distinctive feature matches that route an object to 
its imagp equivalent. CPS searches for goals, it searches for methods to reach those goals, and it accesses 
Information units by tree searching. 
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GPS uses a general technique called "means-ends analysis" to guide the search through the trees. 
Means*ends analysb Involves subdividing the problem Into easier subprobtems. It is accomplt^ed by taking 
differences between w^hat ts given and wliat is desired, ile.» between two objects or between an object and a 
class of objects, or more concretely, the separating of a mathematical problem into simpler addition or 
muttlpllcation problems. Tlius, it is possible to see that the four subgoal routines mentioned are an Integral 
part of the means^ends analysis and die tree^searching process. 

One of the critical questions surrounding GPS and other computer models of problem solvers Is the 
question of memory. Since objects, methods, and goals are all represented by a tree-. * uctured list In GPS, 
essential memoiy slnjcture and process are represented by tree structures and processes, The link from one 
node of a tree to another node is never broken In GPS, Le. , the memoiy is perfect. For example, all node 
Images can be listed, sorted, or restructured without loss. But human thinkinig is surprising interruptable. 
These Interruptions affect memoiy in various ways. Input to memory may be disrupted or Intermittent In 
nature (similar to interniptions while readjjig a novel). Using list structure techniques or stochastic 
principles (Suppes, 1969), representations oPobjects in human memoiy as a tree-structured list can look 
like a tree with many broken limbs. Thus, GPS offers the potential to represent complex memory processes 
associated with reading comprehension. 

The role of personality processes can dso be Incorporated with the GPS model (Simon, 1967a or 
1967b). Simply, personality processes are conceptualized as interrupt commands that switch the GPS 
processor to personality subroutines. 

Robert Shaw (1968) has suggested an Automaton Model of the human memoiy system based on the 
pushdown store. Whfle model is fairly specific to a paticular task, it is suggestive of the possible use of 
automata theoiy In technical training. Taking Into account consistent evidence for performance 
characteristics whkh seem to suggest a short* and long-term nwmoiy, Shaw proposed a memoiy system 
based on two pushdown store transducers illustrated in Figure 20. Each pu^down $tore represents a 
functional component of memoiy. The short-term store (STS) Is a pushdown store which accepts Inputs 
from a perceptual system (PS) and places them in its working noemory. However, the STS has no contr^ 
over the working pushdown memoiy other than the placing of items on the list. It is the long-term store 
(LT$)lWifchhSs IK^ fi^mfem^ of It^mS b^ns^nnh^pOA^^ mm STS tfnd LTS. If irai«^ Ihr LTS 
which transfers information back to STS. Only through SHTS can output be generated to tfie motor system 
(MS). 

The task this model was to account for is the learning of an aperiodfc sequence of symbols. For 
example, a group of sub}ects mi^t receive a string ^f five R symbols followed by five G symbc^s. The 
subjects were to anticipate the next symbol and, when a criterion of two blocks (in this example 5 R*s and 
G*s make a block) was successively anticipated by the 5, and a new block consisting of, for example, seven 
R's followed by seven G's, was presented to criterion. Several variations on this task were used (including 
string-ending markers), but they are not discussed here, 

Shaw defined 48 rules wiiich chaacterized the system by predicting output based on the input 
symbol and the current state of the system (the top of the pushdown store and items below, or the 
history). In this way he was able to determine which rules .Ss used during the task and which ones were not 
used. Further, the current state of S could always be predicted after the first symbol presentation and 
knowling the next input could be predicted. Throu^ the various task variations, Shaw found good 
agreement between the model predictions and the observed data. 

Automata Chanicterizatk)n with Stimulus-Responses Paradigms 

Suppes (1969) has recently attempjed to demonstrate that stimulus-response models of behavior can 
be expressed as automata. The implications of the theorem have been discussed by Suppes for language 
learnJng and the learning of arithmetic algorithms. 

The basis for Suppes' theorem is to show, by utfllzing principles of conditioning, how a person may 
be tau^M with an appropriate reinforcement schedule to respond as an automaton. Suppes points out that 
one of the magor problems Initially was to characterize the Internal states of the automaton. First attempts 
centered around identifying states of the automaton with the conditioning state of the organism. This 
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approach was abandoned because each conditioning state required a corresponding, different automaton, 
live conespondence that was finally chosen was to Identify the responses of the person with the internal 
states. Responses be finitely classed and handled as an automata parameter In effect, interna] states 
and output are welded into the single component of Internal state* 

* Norman Wexler (1970) has continued the automata work at Stanford In language modeling, Wexler 
studied learning of the Japanese language based upon an Automaton Model. The specific model used 
differed from a stock automata requiring no capabflity to erase memory* Instead, a transducer automata 
was used which simply generated an output sequence from input and state without relating input to 
memory, Only one element in memojy was required at any given time as opposed to the total memory of a 
pulldown store. Experiments were run with this model for determining learning sequence of rules. 

Offlr (1971) has also investigated Automata Models at Stanford as part of a set of models of 
individual differences in learning and performance. He particularly was interested in the scheduling of blocks 
and items adaptivelyi Although not directly applicable, his discussion does present relevant questions about 
adapting mathematical models in general to individual differences.- 

Recommendatk>ns 

The following recommendations are made for automata models in AIS: 

• It would be premature at this stage in the deYelq)ment of automata modeling to focus on a 
particular model with its parameters for an application to AIS. Delay is suggested due to the 
theoretical and ^mathematical structure of Automata theory as well a$ the problem of 
representation of states and sequence symbols as discussed earlier;' Therefore, it is 
recommended that this class of model be Identified as a basic research element In AIS and not 
become an operational procedure. 

• This section has attempted to describe two approaches to representation information processing 
and stlmulus^response. Although both of these appear promising, they lack sufficient validation 

1 to allow a firm prediction concerning their value to AIS. It should, perhaps, be reiterated th^t 
what might be gained from these Automata Models is the inclusion of complex human behavior 
into the adaptive system operations of the AIS. Many models are capable: of being used to 
handle complex behavior at a gtoss^level, but fev^ can even attempt an "in-prbcess** modeling. It 
is, therefore, recommended that basic research finding in automata be monitored so as to 
Identify any models that may prove useful In AIS during its development. 

• Given the conceptual and peiformarice complexities found in the training tasks of the three AIS 
courses. Automata Models would be potentially appropriate for each of them. Automata 
Models mi^t be especially useful for the mar^al or semHiterate student, since his learning 
problems and processes could be represented in fine-grained detail and precise remediation 
applied. It is therefore recommended that Automata Models be investigated for their 
applfcation for marginal and semiliterate students. 
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• X. SUMMARY AND CONCLUSIONS 

Thoughtful assessment of the "state-of-the-arl*' and current needs of Adaptive Instmcllonal Models 
reveals the requirement for planning, conceptualizing, and operatlonallzing efforts, rather than for merely 
applied engineering efforts. These needs appear to be best met by a multifaceted research and development 
prograrn. The process of this Research and Development program must parallel (he process reflected In ihe 
AIM iteralure search Accordingly, the first step after this report Is prepared Is to simulate several of the 
models described In this document. These simulations will serve to(l) solidify and make operational the 
concepts and the rnodels, (2) suggest tlie proper use and ranges of model parameters, and (3) provide the 
Aic ?i? V"? Adaptive Model data bas^ and its operation. Once AlM^s are Implemented within the 
i rA if r ^ ^ will continue to evolve and change, given evaluative feedback. This evolutionary process 
must allow for reassessment of current model parameters, the updating of these parameter values, and the 
introduction of new processes and parameters. 

The general conclusions of this report consist of two parts. The first concerns the operational 
conceptualization of AIM's in Sections II through V, which could be sufficiently defined to be 
Implemented In a real system, namely, AIS. Much of the learning model literature haS been concerned with 
speculative models which have never been Implemented and probably never will be, One should recognize 
that, for the AIS, AIM must not only be implemented, but Is the crucial coordinator of the training system. 

The second area of findings concerns a mapping of a domain by which varying models described in 
this report, and those that may develop in the, future from new ideas and techniques, could be further 
Integrated, made operational, evaluated, and modified. The operational conceptualization of the adaptive 
Models for AIS is represented In Figure 21. The models are interrelated In hierarchical fashion according to 
the nature of the tnstmctional events which follow an adaptive decision* The instructional events(Ij) may 
be distinguished along either of two dimensions. The first dimension relates to the type of Instruction 
which will be performed. Ij refers to in t uctlon on a topic, such as power supplies or remedial Instruction, 
ie,, the first level concerns broad, task-determined categories of Instruction. 1^ refers to specific 
instmctlonal lessons such as media presentations or tutorial sequences. Ij is even more specific, referring to 
an individual learning "frame" as in Programmed Instruction or CAI. 

The second relational dimension focuses on lime. In general, as one increases specificity In going from 
It to Ij, one decreases the time duration of the instructional unit. For discussion purposes, U instruction is 
generally 30 minutes or more, whQe I3 instmctiort is typically 1 or 2 minutes at most* 

For each decision level, an adaptive model has been identified in the figure as it relates to the level of 
decision to be made within the instructional process. In some cases, an adaptive model will be proposed for 
more than one decision level. This occurs wherever the nature of the model permits this kind of multiple 
complexity-time application. 

This AIM schema provides the following In'Jghts into the actual and the potential model 
relationships. First, Adaptive Models are Inclusive in the sense that drill-and^pracUce models can be 
incorporated within a dynamic programming modeh Therefore, all the AIM's hierarchically share variables. 
Secondly, the level of task specification and the training event time will determine the specificity of the 
decision and monitoring process. Thus, all AIM's will have similar Intentions as to their adaptive 
approaches, but will differ by task time and decision process variables. Finally, the higlier a model is in the 
hierarchy (e.g., Automata Models), the less likely it is that one can operationally utilize it at tliis time. The 
best future prediction would be a hierarchical integration of all the models. 

The common AIM variables that should be monitored during the evolution of AIS are those described 
in Section I, since these best summarize the relationships among the Adaptive Models presented here. The 
five categories of variables that reflect the operational characteristics ar as follows: 

(1) Task characteristics 

(2) Instructional mode 

(3) Decision processes 

(4) Student characteristics and 

(5) Instructional resources 
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In order to reflect these model characteristics llirough the AIS life cycle, tt will be necessaiy to develop an 
Adaptive Model's frame of reference for tliose responsible for tlie AIS environments One of the functions of 
this document has been to map and portray this frame of reference. 

Recommendations 

In order for AIS and the Adaptive Model component to provide the most cost-effective Individualized 
Instruction, the Adaptive Models will require additional conceptualization and validation. The following 
general tasks arc recommended: 

• Hach of the models described should be simulated to facilitate the specification of initial 
parameters, roles, and decision rules. 

• Each model and its simulation should provide for graduations in number and relational 
complexity of variables so that minimal^ cbmprehensive, and open^nded models may be 
devised/ 

• Each model and the related simulation should allow for variation In parameter values, 
functional relationships, and observable measures so that the models may be revised according 
to AIS cost-effectiveness data. 

• The Air Force should consider spons(<ring a basic research effort, parallel to the AIS effort. Into 
^ the relationships between student characteristics, Instructional parameters, and cost- 

effectiveness parameters. 

Model-specific recommendations are as follows: 

• Drill-and-Practke 

• The two Drill-and-Practice Models, namely, the performance-contingent pacing model and 
the task/frame mastery model, should be simulated and implemented within AIS. 

. • There are many decision points within a Drill-and-Practice Model. Curently, most of these 
decisions will need both conceptual and empirical exploration. It is recommended that 
these variations be incorporated in the AIS research plan. 

• For each review session, a Drill-and-Practice Model should be utilized to formulate the 
problem list, organize the sequence, and provide optimal allocation of practice per 
problem type. All instmctional decisions should be based on individually determined 
parameters. 

0 More specifically for the AIS project, the Drill-and-Practice Model should be utilized in the 
courses as follows: 

• For Inventory Management, the focus should be on coding/index schemes and 
classincatlon stmctures as in security rules; 

• For Precision Measurement Fquipment,^ the major emphasis should be technical and 
concephial aspects of the course; and i 

• For Weapons Mechanics, the focus should be on conceptual factors of electricity and 
image/photo requirements of specific weapons. 

• Adaptive Concept Acquisition Model 

• The proposed Adaptive Concc: i Acquisition (AC A) Mojel should be simulated and 
utimately field tested since it focuses on a prumry requirement of trainire, namely, 
concept acquisition. The topics (/ r-daptive pton^pt'- feedback and task/p^ fformancC' 
related concepts should be inc ^T iorjtcd within ihe ACA Model or becomf correlated 
models. 

• The use of pretask measures ( •» v treasures) ■ » d be cxfcndcd and uHH?'. -i so as to 
increase* th^ f raining efficiencies 
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• The most cosUffectlvc use of media and Instructional resources should become an 
operational component within the ACA model approach. 

• The ACA model should be extensively employed within each of the three AIS courses 
and appropriate evaluative comparisons made. 

Adaptive Ruk-I^mln({ Model 

• The Adaptive Rule-Learning Model should be simulated and ultimately field tested. The 
related, complex process of problem-solving should also be studied should also be studied 
and hopefully integrated into Rule-Learning Model. |^ 

• Although the adaptive model for rute-learning proposed in this report Is based on the 
current theoretical and research Uterature; many features of the model are innovative in 
nature and will require additional research and validation. It^^^^ 

that research U conducted int his area. In addition/ the linear regre^^^ models and 
heuristic decision rules utilized by the adaptive model will require further delineation and 
empirical validation. 

• A simulation of the Adaptive Model will facilitate the ^eclfkation of initial decision 
rules and regression algorilhmi. The input variables and Instructional decisions should be 
expanded or revised according to the constraints and characteristics of the simulated 
adaptive model. It is recommended that two versions of the model be simu!ated-a full 
model and a reduced model. The full version of the model would require a highly 
interactive mode of instruction such as CAI, However, a reduced version of the model 
could be utilized in a seml-interactlve mode such as CMl. Therefore, it Is recommended 
that both versions of the models be simulated. The reduced version would not require an 
updating of the instructional strategy according to within-task performance data. 

• The applicability and extension of the adaptive model for rule-learning into the area of 
problem-sdvlng should be investigated. However, it is anticipated that instructional 
models such as Task Teach might prove to be very useful In problem-sdving applications 
in the AlS. 

• Provision for research into the role and efffcts of the number of examples, prompts, 
types of examples, rule frame structure, and amount and placement of review d\outd be 
made in the implementation of the adaptive model. This research would provide data for 
the revision and refinement of the model. 

Algorithm^ Regressk>ii Model 

• An Algorithmic Regression Model should be developed and simulated since it brklges the 
gap between the description of complex training phenomena and the prescription of 
effectWe, individualized instructional events. It is anticipated that, during the early pha 
of AIS, the Algorithmic Regression Mod^l will be the primary comprehensWe prescriptive 
mechanism. 

• Second, the regression approach appear^ to be e^eclally useful In an tnterlesson context. 
Rather than changing or modilylng the instructional method withbt instmctiOiftl 
session on the basis of responses, a potentially more effective instructional change can be 
made by directing the student to another set of materials within the lesson or to a 
remedial or enhanced lessdn. 

• » Within an operable training system used to capacity, it can be expected that the 

prescription selected by the regression equation as optimal may be unavailable. A 
neces*^ interface to the regression model formulation Is a resource,aHocalion model 
which would provide a list of available resources and estimates of resource use so as to 
permit optimal resource-spares to be ordered. Thus, not only will more efficient use be 
made of available resource, but resources can be maintained at an cfflclent level of supply 
as well. It Is therefore recommended that a resource allocation model be developed as a 
partofAfS. 
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r As a flnaJ recommendation, vdldatlon of the regfcssfon models 

^ a sufficient period of time by the AIS contractor In order to provide sufnclent data to 
update ihe beta weights If required. The contractor should also devise a procedure for 
periodic revalidation and restructuring of the models. 

♦ Dynamk Programming Modd 

• A study Aould be made Into operational feasibility of the development of a Dynamic 
Programming Model for AlS. This study should be based upon performance data 
collected during the Initial phases of the AIS program. As the dynarntc nature of the . 
JndMdualized training In AIS becomes observable and descriptWe in nature, a Dynamic 
Programming Model should be developed. r \ i ^ v 

• Tlic development of the dynymk; programming model Aoul^ be phased as follows 

♦ Initial system description i L 

♦ Single-variable optimization studies 

• Multivariate optimization studies 

• The emerging o^ cost-effective criteria and decision processes with the evc^ved 
dynamic programming model 

• Each of the AIS course operations could profit from the application of a dynamic 
programming model. This Is e^cially the case for PMB and Weapons Mechanics In that 

• the model would provide for optimal utilization of expensive, high-fidelity simulators and 

trainers, 

# Natural Language Model 

• The Natural Language Model described in this report is one of the few in the NLP 
litefature which Is educationally oriented and the only NU* model based on a^^ems 
approach. The model should be researdied further to provide an empirical baris/ The 
funding of such research would not be a long-tc i investment since the state-of-the-art Is 
sufficient to provide a diort-term basis for NLP counseling within AIS, 

• The authoring of NLP dialogues has not been sufficiently recognized as a problem in 
literature. It Is, nevertheless^ a problem directly reUtlng to the i^jecificatlon and 
evaluation of NLP dialogues. The systematic approach of content mapping discussed 
earlier is an attempt toward solution of this problem; Arty educational Iit|t)lementation of 
NLP on more than a resea/ch scale should provide training In thU v«tefn^^^^ 

• A related problem is the validation of the dialogue system. Several possiblitteshave been 
suggested as a result of the work done at Flori^da State University, The first Is to follow a 
standard systems approach relating instruction^ objectives to test questions^ and/thr 

the hierarchical content map, to categories or levels of the hlerardiy, A pretest/posttest 
procedure would probably suffice for data collection. The second possibility Is to use a 
sorting or categorization task as the testing situation within the same system approach. 
This would provide needed information concerning memory organization. 

• In respect to the AIS courses. It Is recommended that the NLP model be Implemented 
and evaluated^ especially In the student evaluation and counseling requirement. 

• Automaton Model , 

• It would be premature at this stage in the development of automata modeling to focus on 
a particular model with its parameters for an application to AIS. Delay is requested due 
to theoretical and mathematical structure of Automata theory as well as the problem of 
representation of states and sequence symbols as discussed earlier. Therefore, it Is 
recommended that this class of mode! be identified as a basic research elementaiy AIS 
and not become an operational procedure. 
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This section has attempted to dcs i i c two approaches to tepresentation-lnformatlon 
processing and stimulus-response, R >m of these appear promising, but they lack sufficient 
validation to allow a firm prediction concerning their value to AIS. It should perhaps be 
reiterated that what miglit be gaiu?u from these Automata Models Is the Inclusion of 
complex human behavior into Uie id jpllve system operations of the AIS. Many models 
are capable of being used to handle complex behavior at a gross level, but few can even 
attempt an "in-process*' modeling It 1$. therefore, recommended that basic research 
findings In automata be monitored foMhe purpose of identifying any models that would 
prove useful in AlS during its developmVftir^ 

Given the conceptual and performance complexities found In the tfajplng tasks of the 
three AIS courses, A\itomata Modeh \vould be potentially appropriate for each of them. 
Automata Models miglu be especiaUy useful for the marginal or semiliterate student i 
since his learning problems and processes could be represented In fine-grained dctaO and 
precise remediation ^^pplted. It iMhcrefore rcconunendcd tha Automata Models be 
investigated for their application to m ifginal and semiliterate students. 



GLOSSARY or TERMS 

Acadmk Counseling -the process of providing factual Information and guidance to students concerning 
progress through courses, potential additional courses, and career fields. 

Adaptive Imtructtoml Model-Ont of many multlfacetcd decision structures which use student characteris- 
tics, Instructional mode parameters, and resource load characteristics In order to assign students to 
appropriate instructional conditions and media. 

Adiustmenf Cbunsellng-'Xht process of providing the student with help zM guidance concerning his own 
personal problems as may be related to his academic and other endeavors. 

vlfr F<?rrt Jf/JcctoWy-A grouping of positions which require common qualincatlons. 
Air Force Specialty Code-k combination of meaningful digltsused to Identify an Air Force Specialty. 
Audiovisual Mec/ii-MedIa utilizing the senses of hearing and sight to encourage or carry on the learning 
process. 

Behavioral Objectlves-lhe specification of the goals of Instruction such that each statement contains an 
Identification of terminal behavior, the minimal Standards for acceptable performance, and the 
conditions under which they are to be performed. 

Cdreer CMWaMW-The process of providing the student with Information concerning his career, both wlthlri 
and out of the Air Force. 

Cdgnitive AciMiles-lhox activities associMed with Uie learning of specific facts, pro^durev decision 
skills, (sec Learning Activities). 

Computer-Assisted Instructhn^k training methods involving man machine interactions "•Ui^*"?^* 

computer to present information to students. The computer Interaction can involve multimedia 

presentations, question-answering, testing, and/or inquiry. 
Cbntputerized Measurement System-k system of evaluating student performance through adaptive-testing 

strategies based prlmarfly on sequence, 
Computer-Mamed Instructton-k method of teaching that utOizes a computer for diagnostic assessment, 

CAI, simulation, counseling, resource allocation, and record keeping. 
Cott Effectiveness-k Measure of the relative effectiveness of a system as related to the cost of the system. 

Effectiveness may be operationally defined within AIS as the continuous operation of the hardware 

components, smooth operation of software development, and 90 percent or more students corn- 
. pletlng AIS courses within 80 percent of current Instructional time. Cost Is operationally defined in 

terms of both fiscal and time parameters. 
Criterion Ob}ecitve-kn objective requiring a tern\inal action or an end product of the student at the 

completion of a unit or lesson. 
Criterion Test-k test designed to measure student attainment of criterion objectives. 
Curricuhm-k specific course(s) of study relating to a particular job specialty. Curriculum Is the all- 

encompassing term for the instructional system software, which includes management and training 

rnaterials, ^s well as media and instructional strategies. 
Dijcussfon Meihod-k method of instruction in which the instructor uses questions to cause students to 

participate actively in a learning situation by exchanging Ideas, opinions, and eJipciienccs to reach 

conclusions that will support learning objectives. 
X>n7/-aAKf-/Vtffr/cc-An instructional technkjue in which words, problems, and pictures, are placed before the 

Student for his dcHnition, Identification, and solution, and which are repeated a number of times, .he 

number of times determined by the performance of the student. 
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Dynamically Generaied-Any IJislnicHonal material or test Item sequence which is supplied online to the 
student Or instructor and which is selected or constfucted as a response to input of the student or 
instructor. 

Educational Technology -Ih^ application of principles of \nodern behavioral science and technology to 
education. 

Enabling Ob}ectlv€-%e Identification of a knowledge or skill that must be learned to permit satisfactory 
achievement of a criterion objective. 

Entering Behavhr-The student's level of knowledge or skill befbre Instruction begins. It refers to his prior 
learning, motivational states, Intellectual ability and development, and cultural and social determi- 
nants of his learning ability. 

Evaluation Plan-k schedule of the activities and/or projects planned to determine the effectiveness of a 
course or group of courses in producing graduates quallP^ to perfonn job re^ 

Feedback to Sfw(/^rt/-Informatlon provided to the student regarding the correctness of his response and 
the quality of his perf6rmance, 

Feedback to 5>'5/fw-lnformatlon obtained from student responses which may be used to make adaptive 
Instructional decisions or to revise the instructional program. 

Formative Evaluallon-Thzi evaluation based on student response data which takes place during ma terJals 
development in order to aid in the revision of the materials to increase their effectiveness (see 
Summative Evaluation). 

Frame- A segment of material which the student handles at one time. It may vary from a few words to a 
full page (or CRT screen) or more, in almost all programming methods, It will require at least one 
response (overt or covert) and provide for knowledge of results before the student proceeds to the 
next frame. 

Ctfwey-Thls Instructional technkjue Involves role playing and decision-making activities In order to develop 
cognitive and affective skills In an often Informal manner. Games dlfifer from a similar technique/ 
simulation, In that rules are utilized to guide the progress of the student through the ga:ne situation 
and the correspondence with *'rear* phenomena Is often greater In the game (see Simulation), 

Hands-on Experience- SxuAenX practice on actual equipment, simulator^ or training aids. 

Hierarchical Analysis- A method of task analysis in which the terminal behavior is separated into a number 
of prerequisite capabilities, forming a hierarchy. 

instructional Sequence Z)ec/r0»i-Those decisions having to do with the determination of thV order or 
succession of instmctlonal units. 

Instructional Strategies-The sexier of decision structures which determine the dynamic nature of Instruc- 
tion. The strategies may Include such as combinations of media selection, pacing, difficulty level, and 
readabflUylevel, 

Instructional System-- An Integrated combination of resources (students, Instructors, materials, equipment, 
and faculties), technkjues, and procedures required to assist the student In achieving speclHed learning 
objectives. * ^ 

Instmctors-lhose personnel who, by training In the specialty field and In the Instmctor training course, are 
qualified to teach ATC Technical Training courses (see Monitors). 

Interactive Testing Mode-lh^i mode of testing which will typically take place at an Interactive computer 
- 0^^ Testing In tliis mode wfll often involve a sequential testing activity in which succeeding test 
Items given to the student are functions of the student's responses to previous test itenif. 

Learner Straiegies-Xhoie algorithms and decision structures which are employed by Jludents for the 
purpose of managing their o^yn Instniction, ^-.^^ > pacing, media selecHons, content decisions, difficulty 
and redundancy level! 
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Mastery-An optimal degree to which content or skills are learned. This may be absolute-- 1 00 percent 
accuracy, or relative-A specified minimum accuracy in a specified number of trials. 

Module- fin entire Instructional unit developed via a systems approach which Includes a specification of 
enabling and criterion objectives, content presentation, and evaluation processes^ 

Moniton -Thc class of instructional personnel which are not qualified as instructors but who keep track of 
necessary inclass training materials, and media, and who supervise students in the absence of a trained 
instructor. Monitors may be considered ^'teachers* aides*' (sec Instructors). 

Motor Skilh-Thosc skills requiring manual dexterity, e.g,, movement involving fingers, arms, and legs. 

Multimedia-Dse of more than one medium to convey the content of instruction. Media available for use 
may include, but need not be limited to texts, programmed instruction, audio and video tapes, slides, 
films, film loops, television, and computers. 

Natural Language Processing-l)\t capability of CAl to interpret student input of natural English and 
respond accordingly, also in natural English. 

Off'line-lhdii part of a computer system which is not under the control of the central processor. 

On-line-lhdii part of a computer system which is under the direct control of the central processor. 

Performance Activities-Those activities specifically related to performance on actual equipment such as 
laboratory work and supervised motor activities. 

Predicted Pacing Paradigm-The set of decislon rules which are a part of the Adaptive Instructional Model 
and which serve to determine optimal pacing requirements on an individualized basis. 

Response Analysh-^lht analysis of a student's response In terms of its correctness or incorrectness, latency, 
or other factors as may be appropriate for the response or for the instructional researcher. 

G?//^c/fert-The recording arjd storing of student resp'6n 

Sft?i/e/acj>J^-nie process by which learning experience^? are ordered to provide effective and efficient 
learning. ^ 

Sequential Testing-kt\ adaptive testing strategy in which the type and number of test items presented to 
each student are variable and are based on the student's response to immediately preceding Items. 

Simulation-K technique in which "real-world" phenomena are mimicked, in an often low-fidelity situa- 
tion, in which costs may be reduced, potential dangers eliminated, and time compressed, The 
simulation may focus on a small subset of the features of the actual real-world situation. Simulation 
differs from another technique, gaming, in that rules and role playing typically do not take place (see 
Games). 

Simulator- Any machine or apparatus that simulates a desired condition or set of conditions. 

Student Coping Behaviors-Those sU2iieg)^e& or activities which students use in meeting or solving a 
particular problem situation. 

Student CWr/gt/e-Student feedback to course developers and administrators concerning a given unit or 
b!ock of instruction. It should provide for a critical systematic evaluation of all phases of the 
instruction including content, presentation, and instruments. 

Summative Evaluation-Thdii evaluation of instructional training materials which takes place in the actual 
classroom environment and which Is designed to provide data for effectiveness analyses. This type of 
evaluation takes place following the major development of the material (see Formative Evaluation 
and Ongoing Evaluation), 

Systems Approach to Instruction-Thc series of procedures \*ich Is employed by educational psycho- 
logists, instructional technolgJsts, and curriculum developers to design and evaluate instructional 
programs. 
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Task -A unit of work activity which forms a significant part of a work assignment done by an Individual, 

Task Analytis-Ihe process of breadking down a particular skill into Us subordinate tasks and subskilU, The 
task analysis serves as a method for tlie definition and Interrelationship of skills necessaiy in the Air 
Force job categories. 
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